HPE Moonshot Networking Cookbook

HPE Moonshot Networking Cookbook
Abstract
This provides overview, setup, configuration, examples and troubleshooting for HPE
Moonshot networking. This document is intended for Hewlett Packard Enterprise Sales,
support engineers, professional partners, system administrators, and others responsible for
the design and deployment of Moonshot into any environment that requires network setup.
Part Number: 801678-007
Published: January 2018
Edition: 9
©
Copyright 2014-2018 Hewlett Packard Enterprise Development LP
Notices
The information contained herein is subject to change without notice. The only warranties for Hewlett
Packard Enterprise products and services are set forth in the express warranty statements accompanying
such products and services. Nothing herein should be construed as constituting an additional warranty.
Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained
herein.
Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession,
use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer
Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government
under vendor's standard commercial license.
Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard
Enterprise has no control over and is not responsible for information outside the Hewlett Packard
Enterprise website.
Acknowledgments
UNIX® is a registered trademark of The Open Group.
Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
microSD is a trademark or a registered trademark of SD-3D in the United States, other countries of both.
Contents
Quick start............................................................................................... 8
Quick start introduction................................................................................................................. 8
Planning the connections..............................................................................................................8
Steps for connecting..................................................................................................................... 9
Introduction........................................................................................... 11
Purpose....................................................................................................................................... 11
Organization................................................................................................................................ 11
Naming conventions................................................................................................................... 12
Examples.................................................................................................................................... 12
Related documentation............................................................................................................... 12
System overview................................................................................... 13
System overview.........................................................................................................................13
HPE Moonshot System...............................................................................................................13
External connectivity................................................................................................................... 14
External connectivity........................................................................................................ 14
Network uplink module external connectivity................................................................... 15
Internal connectivity.................................................................................................................... 16
Switch and uplink modules......................................................................................................... 17
HPE Moonshot-45G Switch Module.................................................................................18
HPE Moonshot-180G Switch Module...............................................................................18
HPE Moonshot-45Gc Switch Module...............................................................................19
HPE Moonshot-45XGc Switch Module............................................................................ 19
HPE Moonshot-180XGc Switch Module.......................................................................... 20
HPE Moonshot-6SFP Uplink Module............................................................................... 20
HPE Moonshot-4QSFP+ Uplink Module.......................................................................... 21
HPE Moonshot-16SFP+ Uplink Module........................................................................... 21
Switch feature comparison table.................................................................................................21
Hardware compatibility................................................................................................................23
Support matrix for switches and uplinks...........................................................................23
Support matrix for cartridges and switches...................................................................... 23
Identifying your hardware and firmware......................................................................................24
Example: Identify the network hardware and firmware through the chassis manager..... 24
Switch Command Line Interfaces............................................................................................... 26
Common command comparison................................................................................................. 26
Common commands........................................................................................................ 26
Users and user interface.................................................................................................. 27
File and firmware management........................................................................................27
General troubleshooting tips....................................................................................................... 28
Network management...........................................................................29
Network management overview..................................................................................................29
Physical management ports........................................................................................................29
HPE Moonshot 1500 Chassis Management Module physical management ports.......... 29
Network uplink module serial console port (RJ-45) physical management ports.............30
Contents
3
Management cables....................................................................................................................31
Virtual serial ports....................................................................................................................... 31
Example: Setting up a switch VSP session (Moonshot-45G/180G Switch)..................... 31
Example: Setting up a switch VSP session (Moonshot-45Gc/45XGc/180XGc Switch)... 32
Example: VSP to a cartridge and remote SSH to chassis manager IP............................ 32
Switch management IP............................................................................................................... 32
Example: Changing the management IP address............................................................33
Telnet/SSH.................................................................................................................................. 34
Example: Enabling Telnet server over management interface (Moonshot-45G/180G
Switches)..........................................................................................................................35
Example: Enabling Telnet server over management interface (Moonshot-45Gc/
45XGc/180XGc Switch)....................................................................................................35
Example: Set up SSH server over management interface (Moonshot-45G/180G
Switches)..........................................................................................................................36
Example: Set up SSH server over management interface (Moonshot-45Gc/45XGc/
180XGc Switch)................................................................................................................36
Updating the firmware.................................................................................................................37
Example: Moonshot-45G/180G switch firmware update..................................................38
Example: Moonshot-45Gc/45XGc/180XGc switch firmware download - TFTP, SCP,
FTP/SFTP........................................................................................................................ 39
Example: Moonshot-45Gc/45XGc/180XGc switch firmware update................................ 40
Moonshot-45Gc/45XGc/180XGc switch firmware update through iLO CM..................... 42
How to free up flash space on a Moonshot 45Gc/45XGc/180XGc switch....................... 43
Troubleshooting IP address setup.............................................................................................. 43
Example: Moonshot-45G/180G PING test....................................................................... 43
Example: Moonshot-45Gc/45XGc/180XGc PING test..................................................... 43
Example: Chassis Manager PING test.............................................................................44
Adapters, cables, and splitters............................................................ 46
Adapters, cables, and splitters overview.....................................................................................46
Adapter types and their uses...................................................................................................... 46
Port options for SFP/SFP+ uplink modules.................................................................................47
Port options for QSFP+ uplink modules......................................................................................49
Troubleshooting adapters, cables, and splitters..........................................................................51
Connectivity and port setup (production network)............................53
Connectivity and port setup overview......................................................................................... 53
Port naming conventions............................................................................................................ 53
45G and 45Gc port naming convention with -6SFP uplink module..................................53
180G port naming convention with -4QSFP+ uplink module........................................... 54
180G port naming convention with -16SFP+ uplink module............................................ 56
45XGc port naming convention with -4QSFP+ uplink module......................................... 58
45XGc port naming convention with -16SFP+ uplink module.......................................... 59
180XGc port naming convention with -4QSFP+ uplink module....................................... 61
180XGc port naming convention with -16SFP+ uplink module........................................ 62
Configuring uplinks..................................................................................................................... 64
Example: Moonshot-45G switch with 6x10G SFP/SFP+ uplink module.......................... 64
Example: Moonshot-180G switch with 4x40G QSFP+ uplink module............................. 64
Example: Moonshot-180XGc switch with 16x10G SFP+ uplink module.......................... 64
Example: Moonshot-180G switch with 4x40G QSFP+ uplink module............................. 65
Example: Moonshot-45XGc/180XGc switch with 4/40G QSFP+ uplink module.............. 65
Port commands...........................................................................................................................66
Example: Show downlink and uplink ports: Moonshot-45G switch.................................. 66
4
Contents
Example: Show downlink and uplink ports: Moonshot-180G Switch Module with the
Moonshot-4QSFP+ Uplink Module...................................................................................68
Example: Show downlink and uplink ports: Moonshot-180G Switch Module with the
Moonshot-16SFP+ Uplink Module................................................................................... 71
Example: Show 1 GbE downlink ports: Moonshot-45Gc switch...................................... 74
Example: Show 10GbE downlink and uplink ports: Moonshot-45XGc switch..................75
Example: Display 10GbE uplink ports: Moonshot-45Gc switch....................................... 77
Example: Display 40 GbE uplink ports: Moonshot-45XGc/180XGc switch...................... 78
Example: Show 10 GbE downlink and uplink ports: Moonshot-180XGc switch...............78
Troubleshooting port setup......................................................................................................... 80
Network loops....................................................................................... 81
Network loop overview................................................................................................................81
What is a network loop?..............................................................................................................81
When is a loop topology OK?..................................................................................................... 81
Uncontrolled loops - what happens?...........................................................................................81
Situations to avoid in Moonshot configurations...........................................................................82
Example: Multiple links.....................................................................................................83
Example: Redundant links............................................................................................... 83
Example: Splitter cable.................................................................................................... 83
Preventing broadcast storms...................................................................................................... 84
Example: Preventing loops by creating a Link Aggregation Group..................................84
Example: Preventing loops by creating VLANs to isolate broadcast domains.................84
Network installation.............................................................................. 86
Network installation overview......................................................................................................86
Creating a PXE server................................................................................................................ 86
Example: Basic network configuration............................................................................. 86
Example: Configure DHCP service.................................................................................. 87
Example: TFTP service configuration.............................................................................. 87
Example: HTTP server for OS installation files................................................................ 88
Example: Preparing OS installation files for HTTP server............................................... 88
Example: PXE boot files...................................................................................................90
Modifying PXE for the Moonshot serial console......................................................................... 91
Example: Modifying your PXE configuration.................................................................... 91
Adding the PXE boot files from your Linux distributions............................................................. 92
Configuring the server.................................................................................................................92
Automating your Linux installation.............................................................................................. 93
Example: Kickstart file for RHEL...................................................................................... 93
Example: Modifying PXE configuration file for an RHEL kickstart................................... 94
Troubleshooting network installation...........................................................................................95
Spanning Tree Protocol........................................................................97
Spanning Tree Protocol overview............................................................................................... 97
What is Spanning Tree Protocol?............................................................................................... 97
How STP works...........................................................................................................................97
Types of Spanning Tree Protocols.............................................................................................. 97
Setting Spanning Tree mode...................................................................................................... 98
Troubleshooting STP setup.........................................................................................................99
VLANs.................................................................................................. 100
VLAN overview..........................................................................................................................100
Contents
5
Isolating cartridges to Network A versus Network B................................................................. 100
Example: VLAN - untagged in, untagged out, C1 to Network A switch, C2 to
Network B switch (Moonshot-45G/180G).......................................................................101
Example: VLAN - untagged in, untagged out, C1 to Network A switch, C2 to
Network B switch (Moonshot-45Gc/45XGc/180XGc).....................................................102
No tagging.................................................................................................................................103
Example: VLAN - untagged in, tagged out, C1 to Network A (Moonshot-45G/180G)....104
Example: VLAN - untagged in, tagged out, C1 to Network A (Moonshot-45Gc/
45XGc/180XGc)............................................................................................................. 105
Port mirroring on a Moonshot switch............................................... 107
Port monitoring overview.......................................................................................................... 107
Setting up port mirroring........................................................................................................... 107
Example: Set up port mirroring on Moonshot-45G/180G switches................................ 107
Example: Set up local port mirroring on Moonshot-45XGc/180XGc switches............... 108
Removing port mirroring............................................................................................................109
Example: Removing port mirroring on Moonshot-45G/180G switches.......................... 109
Example: Removing port mirroring on Moonshot-45XGc/180XGc switches.................. 109
High availability networking............................................................... 111
High availability networking overview........................................................................................ 111
Switch stacking and IRF fabrics................................................................................................ 111
Example: Stacking the Moonshot-45G/180G switch modules........................................112
Example: Creating a stacking or Intelligent Resilient Framework fabric with multiple
Moonshot-45XGc/180XGc switch modules.................................................................... 113
Example: Creating an Intelligent Resilient Framework fabric with Moonshot-45XGc/
180XGc switch modules................................................................................................. 118
Tips and more information on switch stacking................................................................119
Switch and server cartridge LAGs............................................................................................ 120
Server cartridge LAG setup............................................................................................120
Example: Configuring link aggregation on RHEL (+CentOS, Fedora, and so
on)....................................................................................................................... 121
Example: Configuring link aggregation on Ubuntu.............................................. 122
Example: Configuring link aggregation on SLES................................................ 123
Switch LAG setup...........................................................................................................124
Example: Creating static LAG with Moonshot-45G/180G switches.................... 125
Example: Creating static LAG with Moonshot-45XGc/180XGc switches............ 126
Example: Creating dynamic LAG with Moonshot-45G/180G switches............... 127
Example: Creating dynamic LAG with Moonshot-45XGc/180XGc switches....... 127
Tips and more information on switch LAG setup............................................................128
PXE support when LACP is configured...............................................................129
Troubleshooting server cartridge LAG setup..................................................................129
Support and other resources.............................................................130
Accessing Hewlett Packard Enterprise Support....................................................................... 130
Accessing updates....................................................................................................................130
Customer self repair..................................................................................................................131
Remote support........................................................................................................................ 131
Warranty information.................................................................................................................131
Regulatory information..............................................................................................................132
Documentation feedback.......................................................................................................... 132
6
Contents
Acronyms and abbreviations.............................................................133
Contents
7
Quick start
Quick start introduction
To ensure success in your Moonshot network setup, use the following two sections, "Planning the
connections" and "Steps for connecting," to get up and running. The step-by-step instructions include
links to other helpful areas in this guide.
Planning the connections
Before the Moonshot chassis arrives, plan the cable routing and adapters required for the configuration.
Because Moonshot integrates network switches into the chassis, you must have a plan for connecting the
uplink ports into the rest of the network.
Moonshot offers two different types of uplink modules; one based around QSFP+ connections and one
based on SFP+ connections. Both can support direct-attached cables or fiber cables with
transceivers. Depending on what networking infrastructure you are connecting the Moonshot chassis to,
you must select the appropriate connection.
The following information indicates the requirements for connecting the uplinks, either 40GbE or 10GbE,
to a Hewlett Packard Enterprise switch or third-party switch.
Hewlett Packard Enterprise switch
Hewlett Packard Enterprise recommends using either copper or fiber connections.
Connection
40GbE to 40GbE fiber
40GbE to 40GbE copper
40GbE to a single 10GbE fiber
40GbE to single 10GbE copper
Required items
•
2 HPE BLc 40G QSFP+ MPO SR4 Transceiver
•
1 HPE Premier Flex MPO/MPO OM4 8f 10m
Cable
1 HPE X240 40G QSFP+ QSFP+ Xm DAC Cable
(X = 1, 3, or 5)
•
1 HPE BLc 10G SFP+ SR Transceiver
•
1 HPE QSFP/SFP+ Adapter Kit
•
1 HPE Xm Multi-mode OM3 LC/LC FC Cable (X
= 1, 2, or 5)
•
1 transceiver for the other switch
1 HPE X240 QSFP+ 4x10G SFP+ Xm DAC Cable
(X = 1, 3, or 5)
Third-party switch
Hewlett Packard Enterprise recommends using fiber connections.
8
Quick start
Connection
40GbE to 40GbE
40GbE to a single 10GbE
Required items
•
2 BLc 40G QSFP+ MPO SR4 Transceiver
•
Premier Flex MPO/MPO OM4 8f 10m Cable
•
1 BLc 10G SFP+ SR Transceiver
•
1 QSFP/SFP+ Adapter Kit
•
1 Xm Multi-mode OM3 LC/LC FC Cable (X = 1,
2, or 5)
•
1 transceiver for the other switch
For more information, see "Adapters, cables, and splitters."
Steps for connecting
Step
Notes
1. Configure serial access.
Use either the physical serial port or the virtual
serial port through the Chassis Manager. For more
information, see "Network management on page
29.
2. Ensure that the firmware is up-to-date.
For information, see Identifying your hardware
and firmware on page 24.
3. Configure the uplink ports.
Due to the variety of ways to connect to external
networking equipment, some setup might be
required prior to connecting the cables. If you are
not using 40GbEthernet for the QSFP+ module or
10GbEthernet for the SFP+ module, you must
perform additional configuration steps. For more
information, seeConfiguring uplinks on page
64.
4. Configure existing network equipment.
Depending on what you are connecting the
Moonshot chassis to, you might need to perform
additional steps to configure your specific
connection. Contact your local Network
Administrator for more details.
CAUTION:
Before connecting the Moonshot switch to
your existing network environment, be sure
to avoid network loops. For more
information, see Network loops on page
81.
Table Continued
Steps for connecting
9
Step
Notes
5. Connect equipment and verify the switch uplink
connection.
With everything configured correctly, you should be
able to safely connect the physical cables that
connect the pieces of network equipment. If set up
properly, the uplink module link LED illuminates
green. You can verify that the link is established
using the switch CLI.
To determine the status of the uplinks, see Port
commands on page 66.
6. Verify the cartridge-to-switch downlink
connections.
If the uplink to the network is working, verify that
the downlinks are connected. Use the switch CLI to
verify the switch downlink status. For more
information, see Port commands on page 66.
Verify the link status of the cartridge from the
cartridge OS.
After the uplink and downlinks are established, you can begin using the Moonshot cartridges and proceed
with OS deployment or configure more advanced switch features. If you have followed the previous steps
and the uplinks and downlinks are established, but you are unable to access network services, it is
usually a software configuration issue. For more information on advanced switch features and OS
deployment, see the supporting documents and videos or the following sections in this document:
10
•
Spanning Tree Protocol on page 97
•
VLANs on page 100
•
Port mirroring on a Moonshot switch on page 107
•
High availability networking on page 111
•
Switch stacking and IRF fabrics on page 111
Quick start
Introduction
Purpose
The purpose of this HPE Moonshot Networking Cookbook is to provide users of the HPE Moonshot
System with a better understanding of the concepts and steps required when connecting and setting up
the Moonshot System into an existing network, as well as some practical tips and examples in using the
Moonshot network.
The scenarios in this Cookbook are intended to create building blocks to using the Moonshot network
solution. More complex topics will be addressed in white papers or single topic papers.
This document is not meant to be a complete or detailed guide to Moonshot or Moonshot networking, but
is intended to provide the reader with valid examples of how Moonshot networking can be set up and
used in data centers or other networking environments.
Organization
This document is organized as follows:
•
Quick start —Get started with your Moonshot System right away.
•
Introduction —Basic information about this document.
•
System overview —Description of the system components relative to networking. Basic connectivity,
external ports, cartridges, part numbers, documentation locations, and how to tell what you have.
•
Network management —Connecting to and setting up the management features of your Moonshot
System networking. A brief look at the management on Moonshot network switches, physical
management ports versus virtual serial ports.
•
Adapters, cables, and splitters —Connecting your Moonshot to the external world. This section
defines what equipment is needed to connect Moonshot networking, how to find the equipment, and
some tips and troubleshooting items.
•
Connectivity and port setup (production network) —Port naming conventions and how to connect
your Moonshot externally.
•
Network loops —Understanding network loops and how to avoid them within the Moonshot System.
•
Network installation —The basics of setting up your Moonshot in a PXE environment.
•
Spanning Tree Protocol —What is STP and how to setup your Moonshot in your network
environment.
•
VLANs —Setting up and using VLANs with Moonshot.
•
Port mirroring on a Moonshot switch —Port mirroring setup, tips, and examples with Moonshot.
•
High availability networking —Creating Static and Dynamic LAGs in your Moonshot System with
server and switch examples.
•
Switch stacking and IRF fabrics —The basics of setting up stacking with Moonshot.
Introduction
11
Naming conventions
•
Moonshot-45G/180G: The "G" refers to the Hyperscale Family.
•
Moonshot-6SFP: The number refers to the number of uplink ports. The type of connector follows.
•
Moonshot-4QSFP+: The number refers to the number of uplink ports. The type of connector follows.
•
Moonshot-45Gc/45XGc/180XGc: The "Gc" refers to Hyperscale+Enterprise Family.
•
Moonshot-45XGc/180XGc: The "X" refers to 10 Gb connections to server nodes.
Examples
The examples in this document are notated throughout this document and will be titled with "Example:" in
the heading. Commands may be different due to multiple firmware stacks and many types of switches. In
these cases, they are notated as such. These cookbook examples are written by the Hewlett Packard
Enterprise development team for Moonshot and are often notated with special comments, tips, and so on.
Examples are meant to be building blocks to help you get the most common networking issues resolved.
Each example may contain the following information:
•
Title that begins with "Example:"
•
Description/tips: Description of the example, helpful tips, warnings, and notes from the author.
•
Prompts in monospace type
•
User entries in bold monospace type
•
Comments and directions in standard type
•
Other resources and references for this topic
Related documentation
The following table shows related documentation links for the Moonshot System.
Document
Description
Location
Product QuickSpecs
Moonshot System QuickSpecs
Hewlett Packard Enterprise
website (http://www.hpe.com/
info/qs)
Moonshot Documentation Library All Moonshot System public
documents in the Hewlett
Packard Enterprise Information
Library
Hewlett Packard Enterprise
website (http://www.hpe.com/
info/moonshot/docs)
Moonshot networking documents All Moonshot System networking
documents
Hewlett Packard Enterprise
website (http://www.hpe.com/
info/moonshot/networking)
Moonshot videos
12
Naming conventions
Videos related to the setup of the Hewlett Packard Enterprise
Moonshot system
website (http://www.hpe.com/
info/moonshot-videos)
System overview
System overview
This section helps you identify the components in your Moonshot system related to networking, as well as
a comprehensive way to identify your switches, uplink modules, and their features. You can find videos
created for this section on the Hewlett Packard Enterprise website.
HPE Moonshot System
The Moonshot System components are identified below. The Moonshot networking is integrated into the
chassis. Each cartridge SOC has dual-port NICs that interface to the switches. The switches interface to
the uplink modules. The uplink modules pass the traffic out of the chassis. The system can have 1 or 2
switches.
Internal components
Item
Description
1
2x Low-latency switch module
2
45x Hot-plug cartridge
Rear chassis components
System overview
13
Item
Description
1
Serviceable, redundant, dual-rotor, hot-plug fan
modules
2
Network uplink modules
3
Moonshot 1500 Chassis Management Module
4
Common-Slot Power Supplies
This document does not cover the full setup and maintenance of the Moonshot System. For more
information on the Moonshot System, see the Hewlett Packard Enterprise website.
External connectivity
The following figures and tables show the external ports on the Moonshot System.
External connectivity
14
External connectivity
Item
Description
Label
1
iLO CM management port
(Ethernet)
iLO/MGMT
2
iLO CM link port (disabled by
default)
LINK
3
HPE Moonshot 1500 CM module RCM
HPE APM port
4
Moonshot 1500 CM module
diagnostic port
DEBUG
5
iLO CM management serial port
—
6
Moonshot 1500 CM module USB —
connector
7
Moonshot 1500 CM module
microSD slot
—
For more information on the Moonshot 1500 Chassis management module:
•
See the chassis documentation on the Hewlett Packard Enterprise website.
•
See the product QuickSpecs on the Hewlett Packard Enterprise website.
Network uplink module external connectivity
Item
Description
1
Serial console port (RJ-45)
2
SFP+ ports X1 – X61
1 The amount and type of uplink ports varies depending on the model.
For more information on the network uplink module:
Network uplink module external connectivity
15
•
See the chassis user and maintenance guide on the Hewlett Packard Enterprise website (http://
www.hpe.com/info/moonshot/docs).
•
See the product QuickSpecs on the Hewlett Packard Enterprise website (http://www.hpe.com/info/
qs).
Internal connectivity
The following diagram illustrates how Switch Module A and Switch Module B are oriented in the
Moonshot enclosure and how they connect to the cartridges.
The uplink modules take the outside server connections, routed through an uplink module (A or B), and
then to their corresponding switch.
The following illustration shows connectivity from the cartridges to one of the network switches. Each
green line represents up to four NIC connections (one NIC per SOC) per cartridge. Another set of four
NIC connections (one NIC per SOC) per cartridge goes to the other network switch. In total, there are
eight NIC connections per cartridge with four going to each switch. The second NIC port from each
cartridge connects to the other network switch.
Item
Description
1
Network switches
2
16x 10GbE lanes to uplink ports
3
Moonshot cartridges
4
4x 1GbE/10GbE lanes to cartridge ports
The relationship of the NIC ports to the switches and uplink modules is shown in the following figure. The
arrow indicates the front of the chassis.
16
Internal connectivity
Item
Description
1
Switch module A
2
System backplane
3
Switch module B
4
eth0
5
eth1
6
Midplane assembly
7
Uplink module A
8
Uplink module B
Switch and uplink modules
The Moonshot System supports multiple switch modules and uplink modules. This section provides
information about each supported switch module and uplink module.
Switch and uplink modules
17
HPE Moonshot-45G Switch Module
Part numbers for this component are as follows:
•
HPE Moonshot-45G Switch Module part number 704644-B21
•
Spare part number 712675-001
For more information about the Moonshot-45G Switch Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-180G Switch Module
Part numbers for this component are as follows:
18
•
HPE Moonshot-180G Switch Module part number 704642-B21
•
Spare part number 712692-001
HPE Moonshot-45G Switch Module
For more information about the Moonshot-180G Switch Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-45Gc Switch Module
Part numbers for this component are as follows:
•
HPE Moonshot-45Gc Switch Module part number 786617-B21
•
Spare part number 816239-001
For more information about the Moonshot-45Gc Switch Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-45XGc Switch Module
Part numbers for this component are as follows:
HPE Moonshot-45Gc Switch Module
19
•
HPE Moonshot-45XGc Switch Module part number 704654-B21
•
Spare part number 712695-001
For more information about the Moonshot-45XGc Switch Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-180XGc Switch Module
Part numbers for this component are as follows:
•
HPE Moonshot-180XGc Switch Module part number 786619-B21
•
Spare part number 816239-001
For more information about the Moonshot-180XGc Switch Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-6SFP Uplink Module
Part numbers for this component are as follows:
20
HPE Moonshot-180XGc Switch Module
•
HPE Moonshot-6SFP Uplink Module part number 704646-B21
•
Spare part number 712676-001
For more information about the Moonshot-6SFP Uplink Module, see the Hewlett Packard Enterprise
website.
HPE Moonshot-4QSFP+ Uplink Module
Part numbers for this component are as follows:
•
HPE Moonshot-4QSFP+ Uplink Module part number 704652-B21
•
Spare part number 712694-001
For more information about the HPE Moonshot-4QSFP+ Uplink Module, see the Hewlett Packard
Enterprise website (http://www.hpe.com/info/Moonshot-4QSFP-Uplink).
HPE Moonshot-16SFP+ Uplink Module
Part numbers for this component are as follows:
•
HPE Moonshot-16SFP+ Uplink Module part number 783263-B21
•
Spare part number 784973-001
For more information about the Moonshot-16SFP+ Uplink Module, see the Hewlett Packard Enterprise
website (http://www.hpe.com/info/Moonshot-16SFP-Uplink).
Switch feature comparison table
The following table provides a high-level comparison of the Moonshot switches.
HPE Moonshot-4QSFP+ Uplink Module
21
Feature
Hyperscale
Hyperscale
Moonshot-45G Moonshot
Switch
-180G Switch
Hyperscale+
Enterprise
Hyperscale+
Enterprise
Moonshot
-45Gc Switch
Moonshot-45X Moonshot-180
Gc Switch
XGc Switch
Uplink ports
6 x 10 GbE
4 x 40 GbE or
16 x 10 GbE
6 x 10 GbE
4 x 40 GbE or
16 x 10 GbE
4 x 40 GbE or
16 x 10 GbE
Downlink ports
45 x 1 GbE
180 x 1 GbE
45 x 1 GbE
45 x 1/10 GbE
180 x 1/10 GbE
Uplink
bandwidth
60 Gbps
160 Gbps
60 Gbps
160 Gbps
160 Gbps
Oversubscriptio No
n
1.1:1
No
2.8:1
11.3:1
Memory, flash
2 GB, 128 MB
2 GB, 512 MB
2 GB, 512 MB
2 GB, 512 MB
2 GB, 512 MB
Feature
summary
Layer 2, Layer
3, Routing,
QoS, Stacking,
Management
(CLI, SNMP,
sFlow)
Layer 2, Layer
3, Routing,
QoS, Stacking,
Management
(CLI, SNMP,
sFlow)
Layer 2, Layer
3, Routing,
QoS, IPv6,
OpenFlow 1.3,
Precision Time
Protocol, IRF
stacking,
Management
(CLI, SNMP,
sFlow)
Layer 2, Layer
3, Routing,
QoS, IPv6,
OpenFlow 1.3,
Precision Time
Protocol, IRF
stacking,
Management
(CLI, SNMP,
sFlow)
Layer 2, Layer
3, Routing,
QoS, IPv6,
OpenFlow 1.3,
Precision Time
Protocol, IRF
stacking,
Management
(CLI, SNMP,
sFlow)
Stackable
Yes: 4
Yes: 2
Yes: 8 members Yes: 8 members Yes: 8 members
IRF
IRF
IRF
License keys
None
None
None
None
None
Power (includes 50 W Max
uplink module)
210 W Max
50 W Max
73 W Max
210 W
MAC, VLANs
128 K, 4 K
128 K, 4 K
128 K, 4 K
128 K, 4 K
128 K, 4 K
IP routing table
12 K
12 K
12 K
16 K
16 K
Packet buffer
9 MB
22 MB
9 MB
9 MB
22 MB
Yes, Yes, Yes,
Yes
Yes, Yes, Yes,
Yes
Yes, Yes
Yes, Yes
TRILL, FCoE,
No, No, No, No1 No, No, No, No1 Yes, Yes, Yes,
EVB, OpenFlow
Yes
Full Layer 3,
Full IPv6
1 Enterprise market
22
Hyperscale+
Enterprise
System overview
No, No1
No, No1
Yes, Yes
Hardware compatibility
Network component support varies depending on the other components installed in the system. For more
information, see the Moonshot System Configuration and Compatibility Guide in the Hewlett Packard
Enterprise Information Library.
Support matrix for switches and uplinks
The following table identifies compatibility for Moonshot System switches and uplinks.
Switches
Moonshot-45G Switch
Moonshot-6SFP Uplink Moonshot-4QSFP+
Uplink
6 x SFP/SFP+
4x QSFP+
704646-B21
704652-B21
Moonshot 16SFP+
Uplink
Supported
Not supported
Not supported
Not supported
Supported
Supported
Supported
Not supported
Not supported
Supported
Supported
Supported
Supported
16x SFP+
783263-B21
45 Port 1G
704644-B21
Moonshot-180G Switch
180 Port 1G
704642-B21
Moonshot-45Gc Switch
45 Port 1G
786617-B21
Moonshot-45XGc Switch Not supported
45 Port 1/10G
704654-B21
Moonshot-180XGc
Switch
Not supported
180 Port 1/10G
786619-B21
Support matrix for cartridges and switches
The following table identifies compatibility for Moonshot System cartridges and network switch modules.
•
Supported indicates a shipping configuration.
•
Not supported indicates a nonshipping configuration.
Hardware compatibility
23
Switch
1P cartridge1
1P1cartridge
4P2 cartridge 2
1 GbE
10 GbE
1 GbE
Ex: ProLiant m300 server
cartridge
Ex: ProLiant m710 or
m400 server cartridge
Ex: ProLiant m350, m700,
or m800 server cartridge
Supported3
Not supported
Supported3
Supported
Supported3
Not supported
Supported
Not supported
Supported4
Supported
Moonshot-45 Supported
G Switch
45 Port 1G
704644-B21
Moonshot-18 Supported
0G Switch
180 Port 1G
704642-B21
Moonshot-45 Supported
Gc Switch
45 Port 1G
786617-B21
Moonshot-45 Supported
XGc Switch
45 Port
1/10G
704654-B21
Moonshot-18 Supported
0XGc Switch
180 Port
1/10G
786619-B21
1 1P = 1 processor
2 4P = 4 processors
3 The port will run at 1 G.
4 To get maximum 10 Gb cartridge support, populate front only or rear only of the chassis with 10 Gb cartridges.
Identifying your hardware and firmware
The Chassis Manager can identify the hardware and firmware revisions of your network elements. The
following example includes useful commands that can assist with identification.
Example: Identify the network hardware and firmware through the chassis
manager
This example identifies how to get network hardware, firmware, and management IP information from the
Chassis Manager. Use this information for a review of switch-related items before a firmware update or
addition of cartridges. "ALL" returns all switch information. "SA" returns only switch A. "SB" returns only
24
Identifying your hardware and firmware
switch B. For more information on the show command, see the HPE Moonshot iLO Chassis Management
CLI User Guide.
hpiLO-> show switch info all
Switch A:
Product ID: 704642-B21
UUID: 322F1A5F-85F0-571E-9003-6928C7B7D892
Serial Number: 7C5338000R
Switch Firmware: 2.0.1.2
Cartridge Satellite Firmware: 11/11/2013
Management Status: Cartridge OK.
Power: On
Status: OK
UID State: Off
Remote Management Interface:
IPv4: 16.91.23.10
IPv6: N/A
MAC: 2c:44:fd:e8:2d:2b
Uplink Module A:
Product Name: HP Moonshot-4QSFP Uplink Module
Product ID: 704652-B21
Serial Number: KDDVFAXQV4
Switch B:
Product Name: HP Moonshot-180G Switch Module
Product ID: 704642-B21
UUID: 946FC94C-A5AA-5704-B469-620B2A3A493F
Serial Number: 7C5338000D
Switch Firmware: 2.0.1.2
Cartridge Satellite Firmware: 11/11/2013
Management Status: Cartridge OK.
Power: On
Status: OK
UID State: Off
Remote Management Interface:
IPv4: 16.91.23.102
IPv6: N/A
MAC: 2c:44:fd:e9:6c:34
Uplink Module B:
Product Name: HP Moonshot-16SFP+ Uplink Module
Product ID: 783263-B21
Serial Number: 7C547J001B
hpiLO-> show switch info sa
Switch A:
Product Name: HP Moonshot-180G Switch Module
Product ID: 704642-B21
UUID: 322F1A5F-85F0-571E-9003-6928C7B7D892
Serial Number: 7C5338000R
Switch Firmware: 2.0.1.2
System overview
25
Cartridge Satellite Firmware: 11/11/2013
Management Status: Cartridge OK.
Power: On
Status: OK
UID State: Off
Remote Management Interface:
IPv4: 16.91.23.101
IPv6: N/A
MAC: 2c:44:fd:e8:2d:2b
Uplink Module A:
Product Name: HP Moonshot-4QSFP Uplink Module
Product ID: 704652-B21
Serial Number: KDDVFAXQV4
For more information, see the Moonshot iLO Chassis Management CLI User Guide on the Hewlett
Packard Enterprise website (http://www.hpe.com/info/moonshot/docs).
Switch Command Line Interfaces
The Hyperscale family of switches, Moonshot-45G and Moonshot-180G, have different network operating
systems than the Hyperscale+Enterprise switches, Moonshot-45Gc and Moonshot-45XGc, and
Moonshot-180XGc. As such, the command line interface is different between the two.
Common command comparison
Following is a comparison of the most common commands between Cisco IOS switches, Moonshot-45G/
180G switches, and Moonshot-45Gc/45XGc/180XGc switches.
For the full set of Moonshot Network switch commands, see the HPE Moonshot-45G/180G Switch
Module CLI Command Reference Guide and the HPE Moonshot-45XGc Switch Fundamentals Command
Reference on the Hewlett Packard Enterprise website.
Common commands
Cisco IOS
Moonshot-45G/180G
Moonshot-45Gc/45XGc/180XGc
enable
enable
system-view
show flash
dir
dir
show version
show version
display version
show run
show running-config
display currentconfiguration
show start
—
display savedconfiguration
show history
—
display history-command
show interfaces
show port all
display interfaces brief
erase start
erase startup-config
reset savedconfiguration
more flash:/<
—
more <filename>
Table Continued
26
Switch Command Line Interfaces
Cisco IOS
Moonshot-45G/180G
Moonshot-45Gc/45XGc/180XGc
reload
reload
reboot
write memory
write memory
save
show tech-support
show tech-support
display diagnosticinformation
show
show
display
no
no
undo
end
end
return
exit
exit
quit
erase
erase
delete
copy
copy
copy
Cisco IOS
Moonshot-45G/180G
Moonshot-45Gc/45XGc/180XGc
username
username
local-user
username <name>
privilege <level>
password
username <name> level
<level> password
local-user <name> class
<role>
username <name> password
username <name> password
[HP] local-user <name>
Users and user interface
[HP-luser-X-name]
password
line console 0
line console
user-interface console 0
File and firmware management
Cisco IOS
Moonshot-45G/180G
Moonshot-45Gc/45XGc/180XGc
show flash
dir
dir
show bootvar
show bootvar
display boot
show version
show version
display version
copy tftp://<address>/
<file> <location>
copy tftp://<address>/
<file> <location>
tftp <address> get
<file> <local-file>
copy <location> tftp://
<address>/<file>
copy <location> tftp://
<address>/<file>
tftp <address> put
<file>
Users and user interface
27
General troubleshooting tips
The following tips help you to avoid the most common errors when setting up and using the Moonshot
hardware in a networking environment:
•
Verify that your firmware is up to date. For more information, see the HPE Moonshot Component Pack
documentation on the Hewlett Packard Enterprise website.
IMPORTANT:
The firmware for the Moonshot-45Gc/45XGc/180XGc switches cannot be updated using the
Chassis Manager.
28
•
Check the hardware compatibility. For more information, see the HPE Moonshot System
Configuration and Compatibility Guide on the Hewlett Packard Enterprise website.
•
Use the show all command when preparing for help or to request service. This command is
extremely useful for providing detailed information at the time of request.
General troubleshooting tips
Network management
Network management overview
This section helps you to connect your management subsystem with the Moonshot network switches. You
can find videos created for this section on the Hewlett Packard Enterprise website.
The switch management CLI can be accessed in four ways:
•
•
Directly through the switch serial console port:
◦
The switch console port is a physical RJ-45 serial port on the uplink module.
◦
The Moonshot-45G/180G/45Gc/45XGc/180XGc switches connect to the serial port when
configured to 115200 baud rate, 8 data bits, no parity, 1 stop bit, and no flow control.
Through a Chassis Manager VSP session:
◦
•
•
The Chassis Manager allows access to the switch serial port using VSP.
Through the Chassis Manager management network (out-of-band remote management):
◦
The switch service port has a dedicated Ethernet port connected to the Chassis Manager
management network. An IP address can be either statically or dynamically assigned, and SSH/
Telnet can be enabled.
◦
This is the default setting for remote management and is also the Hewlett Packard Enterprise
recommended method for remote management.
Through the switch production network (in-band remote management):
◦
Access through a connection to any uplink port.
◦
An IP address can be either statically or dynamically assigned, and SSH/Telnet can be enabled.
Physical management ports
The following illustrations identify the physical management ports. You can manage the Moonshot
switches through Chassis Manager, the uplink module connection, or both.
HPE Moonshot 1500 Chassis Management Module physical management
ports
When examples note "connected to chassis manager," connect to the iLO CM management port (MGMT)
or the iLO CM management serial port. The iLO CM management serial port requires a DB9 female
connection.
Network management
29
Item
Description
1
iLO CM management port (iLO/MGMT)
2
iLO CM management serial port
For more information on the chassis management module:
•
See the user and maintenance guide on the Hewlett Packard Enterprise website.
•
See the product QuickSpecs on the Hewlett Packard Enterprise website.
Network uplink module serial console port (RJ-45) physical management
ports
When examples note "connected to switch serial port," this indicates a serial connection through the serial
console port. Connection through the serial console port requires an RJ-45 connection.
30 Network uplink module serial console port (RJ-45) physical management
ports
For more information on the network uplink module:
•
See the chassis documentation on the Hewlett Packard Enterprise website.
•
See the product QuickSpecs on the Hewlett Packard Enterprise website.
Management cables
The following cables are needed to connect to the Chassis Manager serial port and switch serial console
port:
•
Chassis Manager serial port:
◦
•
DB9 null modem cable, female-to-female connector
Switch serial console port:
◦
Serial console cable, DB9 RS232 serial to RJ-45
– Hewlett Packard Enterprise part number 797918-B21
Virtual serial ports
The Chassis Manager allows you to connect the VSP to the switch and perform configuration commands.
The VSP is achieved from the Chassis Manager communicating over Ethernet to the management
controller on the switch management. To manage VSP through Moonshot, use the Chassis Manager
Management Link. After determining the iLO management IP address from the Chassis Manager Serial
Port, you can remote into Chassis Manager, and then connect to the switch via VSP to the switch
management controller.
IMPORTANT:
The Moonshot-45G/180G Switch can only have one serial port active at a time. The console port is
active by default. If VSP is enabled, then the console port is disabled.
Example: Setting up a switch VSP session (Moonshot-45G/180G Switch)
The Chassis Manager allows you to connect the VSP to the switch and perform configuration commands.
The VSP session is achieved when the Chassis Manager communicates with the management control on
the switch management. "all" indicates all network items. "sa" indicates switch A. "sb" indicates Switch
B.
hpiLO-> show switch info all
hpiLO-> set switch vsp on all
hpiLO-> set switch power off all
hpiLO-> set switch power on all
hpiLO-> connect switch vsp sa
Virtual Serial Port Active: SA
Starting virtual serial port.
Press 'ESC (' to return to the CLI Session.
Management cables
31
(routing)->
For more information, see the HPE Moonshot iLO Chassis Management CLI User Guide and
Moonshot-45G/180G Switch User and Maintenance Guide on the Hewlett Packard Enterprise website
(http://www.hpe.com/info/moonshot/docs).
Example: Setting up a switch VSP session (Moonshot-45Gc/45XGc/180XGc
Switch)
The Chassis Manager allows you to connect the VSP to the switch and perform configuration commands.
The VSP is achieved through the Chassis Manager communicating to the management control on the
switch management. "all" indicates all network items. "sa" indicates switch A. "sb" indicates Switch B.
hpiLO-> show switch info all
hpiLO-> connect switch vsp sa
Virtual Serial Port Active: SA
Starting virtual serial port.
Press 'ESC ' to return to the CLI Session.
<HP>
For more information, see the HPE Moonshot iLO Chassis Management CLI User Guide and the
Moonshot-45XGc Switch User and Maintenance Guide on the Hewlett Packard Enterprise website (http://
www.hpe.com/info/moonshot/docs).
Example: VSP to a cartridge and remote SSH to chassis manager IP
VSP is a remote access point to manage cartridge and switch systems within the Moonshot System.
When editing Linux text files, it is best to use an SSH instead of a VSP.
hpiLO-> set node power on c2n1
c2: #Cartridge 2
c2n1: #Node 1 Power on requested
hpiLO-> connect node vsp c2n1
Virtual Serial Port Active: C2N1
Starting virtual serial port.
Press 'ESC (' to return to the CLI Session.
Telnet/SSH
[root@<server> ~]# ssh Administrator@<your.CM.IP.address>
Administrator@<your.CM.IP.address>'s password:
User: Administrator logged-in to <client>.<your
domain>.<exmpl>(<your.CM.IP.address>) at 14:01 Sep 18 2014
iLO Chassis Manager v1.30
CLI Version: 2.2
For more information, see the Moonshot iLO Chassis Management CLI User Guide on the Hewlett
Packard Enterprise website.
Switch management IP
The management IP address is changed using the following methods.
32 Example: Setting up a switch VSP session (Moonshot-45Gc/45XGc/
180XGc Switch)
Example: Changing the management IP address
On Moonshot-45G/180G Switches, the default for the management IP address is set to DHCP. To set up
static IP addressing using Chassis Manager, see the following example. Note the PXE requirement.
(Routing) >enable
(routing) #serviceport protocol none
Changing protocol mode will reset ip configuration.
Are you sure you want to continue? (y/n)y
(Routing) #serviceport ip xxx.xxx.xxx.xxx 255.255.255.0 xxx.xxx.xxx.254
(routing) #show serviceport
Interface Status............................... Up
IP Address..................................... xxx.xxx.xxx.xxx
Subnet Mask.................................... 255.255.255.0
Default Gateway................................ xxx.xxx.xxx.254
IPv6 Administrative Mode....................... Disabled
Configured IPv4 Protocol....................... None
Configured IPv6 Protocol....................... None
IPv6 AutoConfig Mode........................... Disabled
Burned In MAC Address.......................... 00:24:81:D0:62:1F
On Moonshot-45G/180G Switches, the default for the management IP address is set to DHCP. See the
following example for setting up dynamic (DHCP) IP addressing using Chassis Manager. Note the PXE
requirement.
(Routing) >enable
(Routing) #serviceport protocol dhcp
Changing protocol mode will reset ip configuration.
Are you sure you want to continue? (y/n)y
(Routing) #show serviceport
Interface Status............................... Up
IP Address..................................... xxx.xxx.xxx.xxx
Interface Status............................... Up
IP Address..................................... xxx.xxx.xxx.xxx
Subnet Mask.................................... 255.255.255.0
Default Gateway................................ xxx.xxx.xxx.xxx
IPv6 Administrative Mode....................... Disabled
Configured IPv4 Protocol....................... DHCP
Configured IPv6 Protocol....................... None
IPv6 AutoConfig Mode........................... Disabled
Burned In MAC Address.......................... 00:24:81:D0:62:1F
Example: Changing the management IP address
33
See the following example for setting up dynamic (DHCP) IP addressing and static IP addressing on the
Moonshot-45Gc/45XGc/180XGc switch.
<HP>system-view
[HP] interface m0/0/0
[HP-M-GigabitEthernet0/0/0] ip address 10.10.10.100 255.255.0.0 (Static IP
addressing)
[HP] ip route-static 0.0.0.0 0.0.0.0 10.10.10.1 (Default Gateway IP address is
10.10.10.1)
[HP-M-GigabitEthernet0/0/0] ip address dhcp-alloc (Dynamic IP addressing)
[HP]save
The current configuration will be written to the device. Are you sure? [Y/N]:
y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
[HP] quit
For DHCP, the IP can be obtained by the following command.
<HP>display interface m0/0/0 brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby – standby
Protocol: (s) – spoofing
Interface Link Protocol Main IP Description
M-GE0/0/0 UP UP 15.215.17.133
For more information, see the HPE Moonshot-45XGc Switch Layer 2 – LAN Switching Command
Reference and HPE Moonshot-45G/180G Switch Module Administrator's Guide on the Hewlett Packard
Enterprise website (http://www.hpe.com/info/moonshot/networking).
Telnet/SSH
The SSH protocol specifies how clients securely connect to SSH servers, and the resulting encrypted link
between them. Each SSH session creates a secure channel to send and receive data. SSH is a Telnetlike program for logging into and executing commands on a remote machine, which includes security with
authentication, encryption, and data integrity features. After SSH is connected and authenticated, the
command line interface becomes available.
CAUTION:
Before setting up Telnet/SSH, an authentication mode needs to be configured for VTY login users:
34
Telnet/SSH
•
For Moonshot-45G/180G Switches, see "Section 6: Configuring Security Features" in the HPE
Moonshot-45G/180G Switch Module Administrator’s Guide.
•
For Moonshot-45Gc/45XGc/180XGc Switches, see "Login overview" in the HPE Moonshot-45XGc
Switch Fundamentals Configuration Guide.
CAUTION:
The following examples are for reference only. Keep network security in mind when using these
commands and choosing passwords for Telnet/SSH.
Example: Enabling Telnet server over management interface
(Moonshot-45G/180G Switches)
User:admin
Password:
(Routing) >enable
(Routing) #ip telnet server enable
(Routing) #configure
(Routing)(config) #aaa authentication enable MYLIST none
(Routing)(config) #line telnet
(Routing)(config-telnet) #enable authentication MYLIST
(Routing)(config-telnet) #exit
(Routing)(configure) #exit
(Routing) #write memory
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide and
the Moonshot-45G/180G Switch Module Administrator's Guide on the Hewlett Packard Enterprise website
(http://www.hpe.com/info/moonshot/networking).
Example: Enabling Telnet server over management interface
(Moonshot-45Gc/45XGc/180XGc Switch)
<HP>system-view
System View: return to User View with Ctrl+Z.
[HP]telnet server enable
[HP]line vty 0
[HP-line-vty0]authentication-mode none
[HP-line-vty0]user-role network-admin
[HP-line-vty0]quit
[HP]quit
<HP>save
The current configuration will be written to the device. Are you sure? [Y/
N]:y
Please input the file name(*.cfg)[flash:/tuned_startup.cfg]
Example: Enabling Telnet server over management interface (Moonshot-45G/
180G Switches) 35
(To leave the existing filename unchanged, press the enter key):
flash:/tuned_startup.cfg exists, overwrite? [Y/N]: y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
For more information, see the HPE Moonshot-45XGc Switch Module CLI Command Reference Guide and
the HPE Moonshot-45XGc Switch Module Administrator's Guide on the Hewlett Packard Enterprise
website (http://www.hpe.com/info/moonshot/docs).
Example: Set up SSH server over management interface (Moonshot-45G/
180G Switches)
(Routing) #ip ssh server enable
(Routing) #configure
(Routing) (Config) #aaa authentication enable MYLIST enable
(Routing) (Config) #line ssh
(Routing) (Config-ssh) #enable authentication MYLIST
(Routing) (Config-ssh) #exit
(Routing) (Config) #exit
(Routing) #write mem
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide and
the Moonshot-45G/180G Switch Module Administrator's Guide on the Hewlett Packard Enterprise website
(http://www.hpe.com/info/moonshot/networking).
Example: Set up SSH server over management interface (Moonshot-45Gc/
45XGc/180XGc Switch)
<HP> system-view
System View: return to User View with Ctrl+Z.
[HP]ssh server enable
[HP]public-key local create rsa
The range of public key modulus is (512 ~ 2048).
If the key modulus is greater than 512, it will take a few minutes.
Press CTRL+C to abort.
Input the modulus length [default = 1024]:
Generating Keys...
.......................++++++
......................................++++++
..................++++++++
...++++++++
Create the key pair successfully.
[HP]user-interface vty 0 15
[HP-line-vty0-15]authentication-mode scheme
36 Example: Set up SSH server over management interface (Moonshot-45G/
180G Switches)
[HP-line-vty0-15] protocol inbound ssh
[HP-line-vty0-15]quit
[HP]local-user admin
New local user added.
[HP-luser-manage-admin]password simple password
[HP-luser-manage-admin]service-type ssh
[HP-luser-manage-admin]authorization-attribute user-role network-admin
[HP-luser-manage-admin]quit
[HP]quit
<HP>save
The current configuration will be written to the device. Are you sure? [Y/
N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
For more information, see the HPE Moonshot-45XGc Switch Fundamentals Guide on the Hewlett
Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Updating the firmware
All switches have specific switch firmware:
•
Satellite Controller—The management interface on the switches and cartridges
•
Cartridge Data—Tells the Chassis Manager which switch is installed and its features.
The cartridges and Chassis Manager have similar firmware bits that can be updated with the switches
through the Moonshot Component Pack. Periodically, new component packs that might introduce new
features will be released. Be sure to check back to see if there are any updates or added features that
might be applicable to your product.
You can access released firmware from the Hewlett Packard Enterprise website.
There are different ways to update the switch firmware:
•
Chassis Manager has the ability to flash the Moonshot-45G Switch and Moonshot-180G Switch
firmware by either storing the flash image from a Web server or directly from the Web server.
•
To update firmware using the Chassis Manager, you need an HTTP/HTTPS Web server accessible to
the iLO Chassis Manager MGMT port on the Chassis Manager. You can either upload the firmware to
the Chassis Manager internal storage and then deploy it to the switches or you can deploy it directly
from the Web server. You can update the switches individually ("SA" or "SB") or both sequentially
("SA-B"). The firmware image is a binary file with a .bin file extension.
The Moonshot–45Gc/45XGc/180XGc Switches must be updated using either a tftp, scp, or ftp/sftp server
through the switch management interface. The firmware image is a binary file with a .jpe file extension.
Confirm the type of switch you have installed. See "Identifying your hardware and firmware."
Updating the firmware
37
If a switch is part of an IRF or is stacked, be sure to review the following guidelines:
•
Stacked switch configurations — Be sure to update the firmware on the switch assigned as the master.
•
IRF switch configurations — Be sure to update the firmware for all switches by using "all" in the
command instead of "slot 1." To update firmware, use the commands as follows:
◦
All switches: boot-loader file flash:/Switch_FW_180XGc_T2421.ipe all main
◦
A single switch (slot 1): boot-loader file flash:/Switch_FW_180XGc_T2421.ipe slot
1 main
For more information, see "Example: Moonshot-45Gc/45XGc/180XGc switch firmware update."
For more firmware update information for the Moonshot-45G/180G Switches, see the Hewlett Packard
Enterprise website.
Example: Moonshot-45G/180G switch firmware update
To update firmware using the chassis manager, an HTTP/HTTPS web server accessible to the iLO
Chassis Manager MGMT port on the chassis manager is required. The firmware can be uploaded to the
chassis manager internal storage and then deployed to the switches, or the firmware can be deployed
directly from the web server. Switches can be updated individually ("SA" or "SB") or both sequentially
("SA-B").
Repository method
hpiLO-> add firmware file http://myserver/path/to/firmware.file
Upload complete: Firmware_File_Header_Name
hpiLO-> update firmware Firmware_File_Header_Name sa
Reading file from iLO Chassis Manager repository
Switch A
Updating Firmware File Header Name
Update Status: 100% complete
Update Complete
Web server method
hpiLO->update firmware http://myserver/path/to/firmware.file sa
Reading file from network
Switch A
Updating Firmware File Header Name
Update Status: 100% complete
To complete the switch FW update, the boot code and the CPLD images must be updated. Using the
switch CLI, the CPLD update checks to see if an update must be performed. Log into the switch and use
the following information as an example for updating the boot code and CPLD images:
(Routing) #update bootcode
Updating boot code, please wait a few seconds... Success!
(Routing) #
Always save any configuration changes before the update CPLD command is executed. When the CPLD
firmware is updated, the switch automatically reboots so that the new CPLD firmware is used. The
following example shows the output of the update CPLD command:
(Routing) #update cpld
The CPLD update takes about 10 minutes and the switch will automatically power cycle.
Do you want to continue? (y/n) y
Device #2 Silicon ID is ALTERA04(01)
Device #1 Silicon ID is ALTERA04(00)
38
Example: Moonshot-45G/180G switch firmware update
erasing MAXII device(s)...
erasing MAXII UFM block...
erasing MAXII CFM block...
programming CFM block...
programming UFM block...
verifying CFM block...
verifying UFM block...
DONE
Wait for the switch to power cycle for the updates to take effect. The following example shows the output
of the update CPLD command if no CPLD firmware must be updated:
(Routing) #update cpld
No CPLD update found!
(Routing) #
The command shown in the following example displays information about the currently installed CPLD
firmware code versions. The example also shows the versions available for installation by using the
update CPLD command.
(Routing) #show cpld versions
Management Module Installed CPLD: 0x23
Available CPLD: 0x23
Fabric Module Installed CPLD: 0x0e
Available CPLD: 0x0e
Faceplate Module Installed CPLD: 0x09
Available CPLD: 0x0a
For more information, see the Moonshot iLO Chassis Management CLI User Guide on the Hewlett
Packard Enterprise website (http://www.hpe.com/info/moonshot/docs).
Example: Moonshot-45Gc/45XGc/180XGc switch firmware download - TFTP,
SCP, FTP/SFTP
To update the firmware, you will need an FTP server.
----TFTP – tftp <Server IP> get <Remote Filename> <Local Filename>
----<HP> tftp 127.0.0.1 get Switch_FW_45Gc_45XGc_E2421.ipe local_
Switch_FW_45Gc_45XGc_E2421.ipe
Press CTRL+C to abort.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
38 59.7M 38 22.8M 0 0 187k 0 0:05:25 0:02:04 0:03:21 193k
----SCP – scp <Server IP> get <Remote Filename> <Local Filename>
----<HP> scp serv.<your.domain>.<exmpl> get Switch_FW_45Gc_45XGc_E2421.ipe
Username: username
Connecting to 16.84.189.36 port 22.
The server is not authenticated. Continue? [Y/N]: y
Do you want to save the server public key? [Y/N]: y
user@192.168.1.36's password: password <this field is hidden>
Switch_FW_45Gc_45XGc_E2421.ipe 100% 60MB 711.1KB/s 01:26
Example: Moonshot-45Gc/45XGc/180XGc switch firmware download - TFTP,
SCP, FTP/SFTP 39
----FTP/SFTP – ftp <FTP Server Address> / sftp <FTP Server Address>
----This server uses standard POSIX FTP client syntax. For more information on how to use the FTP client,
see this helpful website (http://www.cs.colostate.edu/helpdocs/ftp.html).
----<HP> ftp your.ftpserver.domain.net
Press CTRL+C to abort.
Connected to fserv.<Your.Domain>.<exmpl> (192.168.1.36).
220 (vsFTPd 2.3.5)
User (serv.net:(none)): username
331 Please specify the password.
Password: password <this field is hidden>
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>dir
227 Entering Passive Mode (192,168,1,36,153,245).
150 Here comes the directory listing.
-rw-r--r-- 1 0 0 62619648 Feb 10 16:51 Switch_FW_45XGc_7.1.045.E2407.ipe
226 Directory send OK.
ftp>get Switch_FW_45Gc_45XGc_E2421.ipe
Switch_FW_45XGc_7.1.045.E2407.ipe already exists. Overwrite it? [Y/N]:y
227 Entering Passive Mode (192,168,1,36,218,185).
150 Opening BINARY mode data connection for Switch_FW_45XGc_7.1.045.E2407.ipe
(62619648 bytes).
226 Transfer complete.
62619648 bytes received in 65.4 seconds (935.4 kbyte/s)
ftp>quit
221 Goodbye.
Example: Moonshot-45Gc/45XGc/180XGc switch firmware update
IMPORTANT:
When updating the firmware for switches within an IRF, be sure to update all switches in the IRF
instead of a single switch. To update all switches, replace "slot 1" with "all" in the following
command: boot-loader file flash:/Switch_FW_180XGc_T2421.ipe all main
After downloading the Comware image file to the switch internal flash memory, install the file using the
bootloader command for a single switch. To update all switches in an IRF, replace "slot 1" with "all" in the
following command.
<HP> boot-loader file flash:/Switch_FW_45Gc_45XGc_E2421.ipe slot 1 main
40
Example: Moonshot-45Gc/45XGc/180XGc switch firmware update
Verifying the file flash: /switch_fw_45gc_45xgc_e2421.ipe on slot 1....Done.
HP Moonshot-45XGc Switch images in IPE:
In the previous example, the final statement will read Moonshot-45Gc Switch when a Moonshot-45Gc
switch is installed.
moonshot-45xgc-cmw710-boot-e2421.bin
moonshot-45xgc-cmw710-system-e2421.bin
This command will set the main startup software images. Continue? [Y/N]:y
Add images to slot 1.
Decompressing file moonshot-45xgc-cmw710-boot-e2421.bin to flash:/
moonshot-45xgc-cmw710-boot-e2421.bin.......Done.
Decompressing file moonshot-45xgc-cmw710-system-e2421.bin to flash:/
moonshot-45xgc-cmw710-systeme2421.bin..................................................Done.
Decompression completed.
You are recommended to delete the .ipe file after you set startup software
images for all slots.
Do you want to delete flash:/switch_fw_45gc_45xgc_e2421.ipe now? [Y/N]:y
The images that have passed all examinations will be used as the main startup
software images at the next reboot on slot 1.
<HP> reboot
Start to check configuration with next startup configuration file, please
wait.........DONE!
Current configuration may be lost after the reboot, save current
configuration? [Y/N]: y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
The file name is invalid(does not end with .cfg)!
This command will reboot the device. Continue? [Y/N]: y
Now rebooting, please wait...
%Feb 11 01:13:02:180 2014 HP DEV/5/SYSTEM_REBOOT: System is rebooting now.
Always save any configuration changes before the cpld update command is executed. When the
CPLD firmware is updated, the switch automatically reboots so that the new CPLD firmware is used. The
following example shows the output of the cpld update command:
<HP>cpld update
CPLD update start .....................
Management module CPLD update: Succeeded!
.............
Fabric and uplink module CPLD update: Succeeded!
Switch is being automatically power cycled!
<HP>
Network management
41
Wait for the switch to power cycle for the updates to take effect. The following example shows the output
of the cpld update command if no CPLD firmware requires to be updated:
<HP>cpld update
CPLD update start ...
Management module CPLD update: Not required and skipped!
.
Fabric and uplink module CPLD update: Not required and skipped!
<HP>
For more information, see the Moonshot-45XGc Switch Fundamentals Command Reference on the
Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Moonshot-45Gc/45XGc/180XGc switch firmware update through iLO CM
To update firmware using the chassis manager, an HTTP web server accessible to the iLO Chassis
Manager MGMT port on the chassis manager is required.
NOTE:
The iLO Chassis Manager must be running firmware version 1.53 or later. Both the A and B switches
must be updated, unless they are part of the same IRF, as described later in this section.
The switch firmware files with the IPE and corresponding BIN extensions must be copied to the web
server. Either file can be specified in the iLO CM command that updates the switch firmware. The iLO CM
validates the BIN file and instructs the switch firmware to retrieve the IPE payload file from the web
server. The following example command shows how to update switch SA by specifying the BIN file
located in the specified directory:
hpiLO-> update firmware http://myserver/path/MOONSHOT_SWITCH_FW-CMW710R2432P02.signed.bin sa
Reading file from network
The switch firmware will be updated now but the new firmware will not
activate until the switch is rebooted. Please save any configuration changes.
Updating Moonshot Switch Firmware Ver 7.1.045, Release 2432P02
After the firmware is updated, save the switch configuration, if needed, and use the iLO CM Virtual Serial
Port to reboot the switch from its command-line interface using the reboot command to activate the new
firmware. The new firmware will not be active until the switch is rebooted.
NOTE:
Powering down the switch with a Chassis Manager power or reset command is not recommended as
doing so will power-cycle a single switch instead of an IRF. In addition, any unsaved configuration
changes will be lost.
When updating an IRF cluster of switches, only update the master. If the current running firmware version
is F2428, updating a subordinate IRF member will result in failure. When the IRF cluster is rebooted from
the switch console using the reboot command, all members will reboot, and the master will update the
remaining switches to the new firmware at that time. If the current running firmware is R2432P02 or later,
any one of the IRF members can be updated. Updating one IRF member causes all IRF members to be
updated to the new firmware. It is not necessary to update more than one IRF member as the IRF master
will ensure that all members are running the same code.
If necessary, updating the CPLDs must be done separately using the cpld update command.
42
Moonshot-45Gc/45XGc/180XGc switch firmware update through iLO CM
How to free up flash space on a Moonshot 45Gc/45XGc/180XGc switch
A firmware upgrade may fail if there is insufficient space to store the image file in the flash directory of the
switch. To free up room, delete old firmware versions, including .bin and .ipe files. Files with the .ipe
extension may be deleted after they are unpacked to the .bin files of that firmware version.
The delete filename command moves a file to the recycle bin and does not free up flash space. The
recycle bin is a folder named .trash in the root directory of the switch. To view files in a recycle bin, use
the command dir .trash in the root flash directory of the switch.
To empty the recycle bin, use the command, reset recycle-bin.
To delete a file permanently without sending it to the recycle bin, use the command, delete /
unreserved filename.
To restore a file from the recycle bin, use the command,undelete filename.
IRF
When IRF is configured, the preceding commands will only work on the master, regardless of which
switch the user is logged into.
To look at the directory on a particular IRF member, use the command cd slot1#flash:, cd
slot2#flash:, and so on. A dir command will then show the files in that member with the slot number
at the top. Deleting files and the recycle bin can be accomplished on each member as described
preceding after the directory is changed.
Troubleshooting IP address setup
The examples in this section can help with network troubleshooting and remote access for Moonshot
manageability.
Example: Moonshot-45G/180G PING test
To use IP PING test from the switch management CLI to verify connectivity to the Chassis Manager IP
address, perform the following.
(Routing) #ping 192.168.2.102
Pinging 192.168.2.102 with 0 bytes of data:
Reply From 192.168.2.102: icmp_seg = 0. time = 898 usec.
Reply From 192.168.2.102: icmp_seg = 1. time = 350 usec.
Reply From 192.168.2.102: icmp_seg = 2. time = 239 usec.
----192.168.2.102 PING statistics---3 packets transmitted, 3 packets received, 0% packet loss
round-trip (msec) min/avg/max - 0/1/2
(Routing) #
Example: Moonshot-45Gc/45XGc/180XGc PING test
Perform the following to use IP PING test from the switch management CLI to verify connectivity to the
Chassis Manager IP address:
<HP>ping 192.168.2.102
Ping 192.168.2.102 (192.168.2.102): 56 data bytes, press CTRL C to break
How to free up flash space on a Moonshot 45Gc/45XGc/180XGc switch
43
56 bytes from 192.168.2.102: icmp_seg = 0 ttl=255 time = 1.257 ms
56 bytes from 192.168.2.102: icmp_seg = 1 ttl=255 time = 1.594 ms
56 bytes from 192.168.2.102: icmp_seg = 2 ttl=255 time = 1.619 ms
56 bytes from 192.168.2.102: icmp_seg = 3 ttl=255 time = 1.637 ms
56 bytes from 192.168.2.102: icmp_seg = 4 ttl=255 time = 1.621 ms
----Ping statistics for 192.168.2.102---5 packet(s) transmitted, 5 packet(s) received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.257/1.546/1.637/0.145 ms
Example: Chassis Manager PING test
Following are a few common steps that can help with network troubleshooting and remote access for
Moonshot manageability.
When configuring a static IP address for the Chassis Manager Management access, set the Subnet Mask
and gateway. When connecting an SSH session, ensure that both systems are on the same gateway.
hpiLO-> show chassis info
Chassis Information:
iLO Chassis Manager: 1.20
CLI Version: 2.2
Chassis Name: ILOCN63170FET
Chassis Manager (switch A-B):
IP: 192.168.2.102 / 255.255.255.0
MAC: a4:5d:36:b7:bc:58
Asset Tag:
Product Name: HP Moonshot 1500 Chassis
Serial Number: USE349M311
Product ID: 747049-B21
UUID: D161FF86-2E9C-5048-8B86-4B59DC367030
Switch A Remote Management Interface:
IPv4: 192.168.2.110
IPv6: Unavailable
MAC: 2c:44:fd:e9:44:ce
Switch B Remote Management Interface:
IPv4: 192.168.2.108
IPv6: Unavailable
MAC: 2c:44:fd:e9:46:01
[root@bemingus ~]# ping 192.168.2.102
PING 192.168.2.102 (192.168.2.102) 56(84) bytes of data.
64 bytes from 192.168.2.102: icmp_seq=1 ttl=255 time=0.934 ms
64 bytes from 192.168.2.102: icmp_seq=2 ttl=255 time=0.297 ms
64 bytes from 192.168.2.102: icmp_seq=3 ttl=255 time=0.253 ms
64 bytes from 192.168.2.102: icmp_seq=4 ttl=255 time=0.318 ms
64 bytes from 192.168.2.102: icmp_seq=5 ttl=255 time=0.328 ms
64 bytes from 192.168.2.102: icmp_seq=6 ttl=255 time=0.368 ms
64 bytes from 192.168.2.102: icmp_seq=7 ttl=255 time=0.309 ms
^C
--- 192.168.2.102 ping statistics --7 packets transmitted, 7 received, 0% packet loss, time 6356ms
rtt min/avg/max/mdev = 0.253/0.401/0.934/0.219 ms
[root@bemingus ~]# ssh Administrator@192.168.2.102
44
Example: Chassis Manager PING test
Administrator@192.168.2.102's password:
User: Administrator logged-in to ILOCN63170FET.(192.168.2.102) at 15:00 Sep 18 2014
iLO Chassis Manager v1.20
CLI Version: 2.1
Type
Type
Type
Type
'HELP' to display a list of valid commands.
'HELP ALL' to display all commands in a tree format.
'HELP <command>' to display detailed information about a specific command.
'HELP HELP' to display more detailed information about the help system.
Network management
45
Adapters, cables, and splitters
Adapters, cables, and splitters overview
This section helps you understand, identify, and order the required components to connect your Moonshot
networking to your infrastructure. You can find videos created for this section on the Hewlett Packard
Enterprise website.
Adapter types and their uses
The following table identifies what types of adapters there are and when to use them.
Adapter
Type
Media
SFP 1 GbE
1000BASE-T
CAT 5/5e/6/7
RJ-45 Connector
SFP 1 GbE
1000BASE-SX
OM 1/2/3/4
LC Connector
SFP+ 10 GbE
10GBASE-SR
OM 1/2/3/4
LC Connector
SFP+ 10 GbE
SFP+ 10 GbE
10GBASE-T
10 G DAC
Pros
Short Range
•
Inexpensive
•
Common
•
Longer Range
•
More Expensive
•
Multi-vendor
support
•
Static Lengths
•
Common
•
Multi-vendor
support
RJ-45 Connector
Uses existing
twisted pair
networks
Integrated
Lower Cost
CAT 5e/6/7
Cons
Expensive
Not Yet Available
•
Short Range
•
Not multi-vendor
compatible
SFP+ 10 GbE
10 G AOC
Integrated
Longer Range than Expensive
DACs
QSFP+ 40 GbE
40 G BASE-SR4
OM3/4
Multi-vendor
support
MPO Connector
QSFP+ 40 GbE
40 G DAC
Integrated
Lower Cost
Expensive
•
Short Range
•
Not multi-vendor
compatible
Table Continued
46
Adapters, cables, and splitters
Adapter
Type
Media
Pros
QSFP+ 40 GbE
40 G AOC
Integrated
Longer Range than
•
DACs
QSFP+ 40 GbE
QSFP+ 40 GbE
4 x 10 G to 40 G
DAC
4 x 10 G to 40 G
AOC
Integrated
Integrated
•
Can connect to
SFP+ ports
•
Lower Cost
•
Can connect to
SFP+ ports
•
Longer Range
than DACs
Cons
Expensive
•
Not multi-vendor
compatible
•
Short Range
•
Not multi-vendor
compatible
•
Expensive
•
Not multi-vendor
compatible
Port options for SFP/SFP+ uplink modules
This section includes common transceivers, adapters, and cables. To identify the components that are
supported for each uplink module and output type, see the QuickSpecs for each module on the Hewlett
Packard Enterprise website.
The transceivers and cables provided in this section are qualified and certified to work with this product.
Transceiver and DAC cables from any manufacturer can be used, but have not been tested with the
product.
•
SFP to 1000 Base SX Transceiver
SFP to LC
HPE 1000Base-SX SFP (mini-GBIC) Module - 1 x 1000Base-SX
Hewlett Packard Enterprise part number 453151-B21
•
SFP to 1000 Base T Transceiver
SFP to RJ-45
HPE 1000BaseT SFP (mini-GBIC) Module - 1 x 1000Base-T
Hewlett Packard Enterprise part number 453154-B21
Port options for SFP/SFP+ uplink modules
47
•
SFP+ to SR Transceiver
SFP+ to SR
A 10-Gigabit transceiver in SFP+ form-factor for single-mode or multi-mode fibers
Hewlett Packard Enterprise part number 455883-B21
•
10GbE SFP+ to SFP+ Direct Attach Copper Cable
SFP+ to SFP+ in multiple lengths.
Hewlett Packard Enterprise part numbers:
48
◦
J9281B – 1m
◦
J9283B – 3m
◦
J9285B – 7m
Adapters, cables, and splitters
For the latest information, see the product QuickSpecs on the Hewlett Packard Enterprise website.
Port options for QSFP+ uplink modules
This section includes common transceivers, adapters, and cables. To identify the components that are
supported for each uplink module and output type, see the QuickSpecs for each module on the Hewlett
Packard Enterprise website (http://www.hpe.com/info/qs).
The transceivers and cables listed are qualified and certified to work with this product. Transceiver and
DAC cables from any manufacturer can be used, but they have not been tested with the product.
•
40GBASE-SR4 Transceiver
QSFP to MPO
Hewlett Packard Enterprise part number JG325B
Comments: Requires 40 GB MPO fiber for this transceiver for use.
•
Fiber Optic Cable used with the QSFP/MPO Transceiver.
HPE Premier Flex MPO/MPO Multimode OM4 8 Fiber 50m Cable
Hewlett Packard Enterprise part number QK731A
Comments: Fragile cable. Use extra precaution when installing and or removing from infrastructure.
Port options for QSFP+ uplink modules
49
•
40GBASE – SFP+ Adapter
Hewlett Packard Enterprise part number 655874-B21
Comments: This turns one 40Gb QSFP+ port into one 10Gb SFP+ port and requires one SFP+
Module to connect. This is just the housing. See above. This 40Gb port must be configured to 4x10Gb
mode. For more information, see Configuring uplinks on page 64.
•
40GbE QSFP+ to QSFP+ Direct Attach Copper Cable
QSFP+ to QSFP+ in multiple lengths.
part numbers:
◦
JG326A – 1m
◦
JG327A – 3m
◦
JG328A – 5m
Comments: Bulky cable
50
Adapters, cables, and splitters
•
40GbE QSFP+ to 4x10G SFP+ Splitter Cable
QSFP+ to 4x SFP+ in multiple lengths.
Does not require the SFP modules.
part numbers:
◦
JG329A – 1m
◦
JG330A – 3m
◦
JG331A – 5m
CAUTION:
Avoid network loops. For more information, see Network loops on page 81.
For the latest information, see the product QuickSpecs on the Hewlett Packard Enterprise website (http://
www.hpe.com/info/qs).
Troubleshooting adapters, cables, and splitters
Following are some tips to help you avoid the most common errors when setting up and using the
adapters, splitters, and so on when networking.
Troubleshooting adapters, cables, and splitters
51
52
•
Avoid network storms. When using the 4x10GbE splitter, be aware of your configuration to avoid
network loops. For more information, see "Network loops."
•
When using the 40GBase to SFP+ adapter, you must also order the SFP/SFP+ module.
•
When using the 40GBase to SFP+ adapter, only one of the ports is available. That is, you will have
10G bandwidth and not 40G because it disconnects the other 3.
•
The fiber cable is fragile. Handle with care.
•
Use static bags and proper ESD protection when using these devices.
•
The Hewlett Packard Enterprise devices listed in this document have been verified to work. Other
devices can be used but warnings might occur and functionality is not verified.
Adapters, cables, and splitters
Connectivity and port setup (production
network)
Connectivity and port setup overview
This section helps you connect your Moonshot ports to external networks. You can find videos created for
this section on the Hewlett Packard Enterprise website.
The Ethernet Switches are top-loaded like the cartridges. These are considered the downlink modules.
The uplinks are in the back of the chassis on each side of the Chassis Manager and above the power
supply units. The switch does not completely boot-up until the uplink modules are installed with the
downlink switch. If there is a mismatch between the Switch Downlink and Uplink Modules, the switch will
power on, but it will not be able to connect. Standing at the rear of the chassis, the uplink module for
Switch A is on the right of the chassis, while Switch B is on the left.
Each node on a cartridge has at least two NICs ports, eth0 for Switch A and eth1 for Switch B. The
switches can either be placed into redundant mode or stacked. For more information, see "Switch
stacking and IRF fabrics," which shows the simplified network connectivity to each switch from a single
and quad node cartridge.
Port naming conventions
This section includes diagrams to illustrate port naming conventions for Moonshot Switch configurations.
45G and 45Gc port naming convention with -6SFP uplink module
The following diagram illustrates how the 45 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-6SFP uplink
module, has 6 uplink ports of 10 Gbps (1/1/1 to 1/1/6) in the backend and 45 downlink ports (1/0/1 to
1/0/45) at 1 Gbps internally to cartridges.
This uplink module can use any of the following:
•
10 Gb SFP+ to SFP+ DAC cable
•
SFP+ to SR transceiver
•
SFP to 1000Base-T transceiver
•
SFP to 1000Base-SX transceiver
Connectivity and port setup (production network)
53
180G port naming convention with -4QSFP+ uplink module
The following diagram illustrates how the 180 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-4QSFP+
uplink module, has 4 uplink ports of 40 Gbps (1/1/1, 1/1/6, 1/1/11, 1/1/16) in the backend and 180
downlink ports (1/0/1 to 1/0/180) at 1 Gbps internally to cartridges. Each of these four QSFP+ ports can
also be configured into 4 x 10 Gb ports.
This uplink module can use any of the following:
54
180G port naming convention with -4QSFP+ uplink module
•
40 Gb QSFP+ cable
•
Splitter cable (QSFP+ to SFP+ 4 x 10 Gb)
•
40 Gb to 10 Gb adapter
Connectivity and port setup (production network)
55
180G port naming convention with -16SFP+ uplink module
The following diagram illustrates how the 180 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-16SFP+
uplink module, has 16 uplink ports of 10 Gbps (1/1/1 to 1/1/16) in the backend and 180 downlink ports
(1/0/1 to 1/0/180) at 1 Gbps internally to cartridges.
This uplink module can use any of the following:
56
•
10 Gb SFP+ to SFP+ DAC cable
•
SFP+ to SR transceiver
•
SFP to 1000Base-T transceiver
•
SFP to 1000Base-SX transceiver
180G port naming convention with -16SFP+ uplink module
Connectivity and port setup (production network)
57
45XGc port naming convention with -4QSFP+ uplink module
The following diagram illustrates how the 45 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-4QSFP+
uplink module, has 4 uplink ports of 40 Gbps (1/1/1, 1/1/2, 1/1/3, 1/1/4) in the backend and 45 downlink
ports (1/0/1 to 1/0/45) at 10 Gbps internally to cartridges. Each of these four QSFP+ ports can also be
configured into 4 x 10 Gb SFP+/SFP ports.
This uplink module can use any of the following:
58
•
40 Gb QSFP+ cable
•
Splitter cable (QSFP+ to SFP+ 4 x 10 Gb)
•
40 Gb to 10 Gb adapter
45XGc port naming convention with -4QSFP+ uplink module
45XGc port naming convention with -16SFP+ uplink module
The following diagram illustrates how the 45 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-16SFP+
uplink module, has 16 uplink ports of 10 Gbps (1/1/1 to 1/1/16) in the backend and 45 downlink ports
(1/0/1 to 1/0/45) at 1/10 Gbps internally to cartridges.
This uplink module can use any of the following:
•
10 Gb SFP+ to SFP+ DAC cable
•
SFP+ to SR transceiver
•
SFP to 1000Base-T transceiver
•
SFP to 1000Base-SX transceiver
45XGc port naming convention with -16SFP+ uplink module
59
60
Connectivity and port setup (production network)
180XGc port naming convention with -4QSFP+ uplink module
The following diagram illustrates how the 180 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-4QSFP+
uplink module, has 4 uplink ports of 40 Gbps (1/1/1, 1/1/2, 1/1/3, 1/1/4) in the backend and 180 downlink
ports (1/0/1:1 to 1/0/45:4) at 1/10 Gbps internally to cartridges. Each of these four QSFP+ ports can also
be configured into 4 x 10 Gb SFP+/SFP ports.
This uplink module can use any of the following:
•
40 Gb QSFP+ cable
•
Splitter cable (QSFP+ to SFP+ 4 x 10 Gb)
•
40 Gb to 10 Gb adapter
180XGc port naming convention with -4QSFP+ uplink module
61
180XGc port naming convention with -16SFP+ uplink module
The following diagram illustrates how the 180 port switches interface to a 1P cartridge. It provides the port
naming convention used by the switch firmware. This switch, when paired with a Moonshot-16SFP+
uplink module, has 16 uplink ports of 10 Gbps (1/1/1 to 1/1/16) in the backend and 180 downlink ports
(1/0/1:1 to 1/0/45:4) at 1/10 Gbps internally to cartridges.
This uplink module can use any of the following:
62
•
10 Gb SFP+ to SFP+ DAC cable
•
SFP+ to SR transceiver
•
SFP to 1000Base-T transceiver
•
SFP to 1000Base-SX transceiver
180XGc port naming convention with -16SFP+ uplink module
Connectivity and port setup (production network)
63
Configuring uplinks
This section provides examples for configuring uplinks depending on the components installed in the
system configuration.
Example: Moonshot-45G switch with 6x10G SFP/SFP+ uplink module
The 6SFP uplink module can configure the uplink ports to either 1G or 10G mode. However, this is not
automatic. To configure an uplink port to run at 1G, you must log in and enter privileged mode. After that,
you must enter the configuration menu, and finally enter the interface menu for the ports you want to
configure. Use the speed command to set the port speed and duplex settings. When completed, exit back
to the root menu and write your settings to memory. A 1000BASE-T or 1000SR transceiver is required to
achieve 1G from the SFP+ uplink module.
Connect through serial console port or VSP.
(Routing)
(Routing)
(Routing)
(Routing)
(Routing)
(Routing)
#configure
(Config)#interface 1/1/1
(Interface 1/1/1)#speed 1000 full-duplex
(Interface 1/1/1)#exit
(Config)#exit
#write memory
To set the port back to 10G mode, use the 10g keyword instead of the 1000 keyword in the previous
example.
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Moonshot-180G switch with 4x40G QSFP+ uplink module
The 4QSFP+ uplink module has the capability of creating a 1G interface from a single 40G connection. To
create a 1G interface from a 40G interface, you need a QSFP+ to SFP+ adapter and a SFP to
1000BASE-SX transceiver or SFP to 1000Base-T (RJ-45) transciever. To configure the interface, log in
and enter into privileged mode. Configure the interface's portmode to operate each of the four 10G
uplinks independently.
IMPORTANT:
This is ONLY for lab environment use. You will only have 1G of the 40G capable bandwidth. This
may only be accessible with a switch bootleg
1. Perform the steps as shown in the following example to convert the 4x40G QSFP+ Uplink Module to
4x10G mode.
2. Reboot the switch.
3. Perform the steps as shown in the previous example to convert one of the 10G interfaces to 1G.
Speed 1000 full-duplex
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website.
Example: Moonshot-180XGc switch with 16x10G SFP+ uplink module
The 16SFP+ Uplink Module can configure its uplinks ports to either 1G or 10G mode. However, this
action is not automatic. To configure an uplink port to run at 1G you will need to log in and enter privileged
mode, then enter the configuration menu, and finally enter the interface menu for the ports you want to
configure. You will use the speed command to set the port speed and duplex settings. When completed,
64
Configuring uplinks
exit back to the root menu and write your settings to memory. A 1000BASE-T or 1000SR transceiver is
required to achieve 1G from the SFP+ uplink module.
Connect through serial console port or VSP.
(Routing)
(Routing)
(Routing)
(Routing)
(Routing)
(Routing)
#configure
(Config)#interface 1/1/1
(Interface 1/1/1)#speed 1000 full-duplex
(Interface 1/1/1)#exit
(Config)#exit
#write memory
To set the port back to 10G mode, use the 10g keyword instead of the 1000 keyword in the previous
example.
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Moonshot-180G switch with 4x40G QSFP+ uplink module
The 4QSFP+ uplink module has the capability of creating four 10G interfaces from a single 40G
connection. To create a 4x10G interface from a 1x40G interface, you need a QSFP+ 10G adapter. To
configure the interface, log in and enter into privileged mode. Configure the interface's portmode to
operate each of the four 10G uplinks independently.
IMPORTANT:
Take care not to create network loops with the 4x10G splitter cable. For more information, see
"Network loops."
Connect through serial console port or VSP.
(Routing) # config
(Routing) (Config)# interface 1/1/1
(Routing) (Interface 1/1/1)# hardware profile portmode 4x10g
This command will not take effect until the switch is rebooted.
(Routing) (Interface 1/1/1)# exit
(Routing) (Config)# exit
(Routing) # reload
Are you sure you want to reload the stack? (y/n) y
After the switch reboots, the interface converts from a single 40G uplink into four individual links. To
convert back from a 4x10G mode to 1x40G, use the 1x40g keyword instead of 4x10g in the previous
example.
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website.
Example: Moonshot-45XGc/180XGc switch with 4/40G QSFP+ uplink
module
The 4QSFP+ uplink module can create four 10 G interfaces from a single 40 G connection. To create a 4
x 10 G interface from a 1 x 40 G interface, you need a QSFP+ 10 G adapter.
To configure the interface:
1. Log in and enter privileged mode.
2. Configure the portmode of the interface to operate each of the four 10 G uplinks independently.
Example: Moonshot-180G switch with 4x40G QSFP+ uplink module
65
The Moonshot-45XGc/180XGc switches do not support 4 x 1 Gb mode.
IMPORTANT:
Take care not to create network loops with the 4x10G splitter cable. For more information, see
Network loops on page 81.
Connect through serial console port or VSP:
<HP>system-view
System View: return to User View with Ctrl+Z.
[HP]interface fortygige 1/1/4
[HP-FortyGigE1/1/4]using tengige
The interface FortyGigE1/1/4 will be deleted. Continue? [Y/N]:y
Reboot the member device to make the configuration take effect.
[HP-FortyGigE1/1/4]quit
[HP]quit
<HP>reboot
To convert uplink from 10 G back to 40 G:
<HP>system-view
System View: return to User View with Ctrl+Z.
[HP]interface
ten-gigabitethernet 1/1/4:2
[HP-Ten-GigabitEthernet1/1/4:2]using fortygige
The interfaces Ten-GigabitEthernet1/1/4:1 through Ten-GigabitEthernet1/1/4:4 will be deleted. Continue? [Y/
N]:y
Reboot the member device to make the configuration take effect.
[HP-Ten-GigabitEthernet1/1/4:2]quit
[HP]quit
<HP>reboot
After the switch reboots, the interface converts from a single 40 G uplink to four individual links. To revert
from a 4 x 10 G mode to a 1 x 40 G mode, use the 1 x 40 G keyword instead of the 4 x 10 G in the
previous example.
After the switch reboots, the interface converts from a single 40 G uplink to four individual 10 G links. The
new 10 G ports are displayed as follows:
1/1/4:1
1/1/4:2
1/1/4:3
1/1/4:4
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Port commands
This section provides examples of port commands depending on the components installed in the system
configuration.
Example: Show downlink and uplink ports: Moonshot-45G switch
This one command displays the downlink and uplink ports for the Moonshot-45G switch with the
Moonshot-6SFP uplink module. This is displayed through the serial console port or the switch VSP.
The 0/3/x ports that follow the 1/1/6 port are not included in this example.
(Routing) #show port all
66
Port commands
Actor
Admin
Physical
Physical
Link
Link
LACP
Type
Mode
Mode
Status
Status
Trap
Mode
--------
--------
--------
--------
--------
--------
--------
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
INTF
Timeout
--------------1/0/1
long
1/0/2
long
1/0/3
long
1/0/4
long
1/0/5
long
1/0/6
long
1/0/7
long
1/0/8
long
1/0/9
long
.
.
.
.
.
.
1/0/40
long
Table Continued
Connectivity and port setup (production network)
67
1/0/41
Enable
Auto
Enable
1000
Full
Up
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
long
1/0/42
long
1/0/43
long
1/0/44
long
1/0/45
long
1/1/1
long
1/1/2
long
1/1/3
long
1/1/4
long
1/1/5
long
1/1/6
long
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Show downlink and uplink ports: Moonshot-180G Switch Module
with the Moonshot-4QSFP+ Uplink Module
The following example shows how to display the downlink and uplink ports for the Moonshot-180G switch
with the Moonshot-4QSFP+ uplink module. This is displayed through the serial console port or the switch
VSP.
The 0/3/x ports that follow the 1/1/20 port are not included in this example.
Actor
Type
Admin
Physical
Physical
Link
Link
LACP
Mode
Mode
Status
Status
Trap
Mode
INTF
Timeout
Table Continued
68 Example: Show downlink and uplink ports: Moonshot-180G Switch
Module with the Moonshot-4QSFP+ Uplink Module
---------
---------
---------
---------
---------
---------
---------
---------
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
Down
Enable
Enable
--------1/0/1
long
1/0/2
long
1/0/3
long
1/0/4
long
1/0/5
long
1/0/6
long
1/0/7
long
1/0/8
long
1/0/9
long
.
.
.
.
.
.
.
.
1/0/175
long
1/0/176
long
1/0/177
long
Table Continued
Connectivity and port setup (production network)
69
1/0/178
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
40G Full
Down
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
40G Full
Down
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
40G Full
Down
EnableEn
able
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
long
1/0/179
long
1/0/180
long
--more-or
(q)uit
1/1/1
long
1/1/2
long
1/1/3
long
1/1/4
long
1/1/5
long
1/1/6
long
1/1/7
long
1/1/8
long
1/1/9
long
1/1/10
long
1/1/11
long
1/1/12
long
1/1/13
long
Table Continued
70
Connectivity and port setup (production network)
1/1/14
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
40G Full
Down
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
Enable
10G Full
Detach
Enable
Enable
long
1/1/15
long
1/1/16
long
1/1/17
long
1/1/18
long
1/1/19
long
1/1/20
long
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website.
Example: Show downlink and uplink ports: Moonshot-180G Switch Module
with the Moonshot-16SFP+ Uplink Module
The following example shows how to display the downlink and uplink ports for the Moonshot-180G switch
with the Moonshot-16SFP+ uplink module. This is displayed through the serial console port or the switch
VSP.
The 0/3/x ports that follow the 1/1/16 port are not included in this example.
(Routing) #show port all
Actor
Admin
Physical
Physical
Link
Link
LACP
Type
Mode
Mode
Status
Status
Trap
Mode
---------
---------
---------
---------
---------
---------
---------
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
INTF
Timeout
----------------1/0/1
long
1/0/2
long
Table Continued
Example: Show downlink and uplink ports: Moonshot-180G Switch Module with
the Moonshot-16SFP+ Uplink Module 71
1/0/3
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
1000
Full
Up
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
Enable
Auto
Down
Enable
Enable
long
1/0/4
long
1/0/5
long
1/0/6
long
1/0/7
long
1/0/8
long
1/0/9
long
.
.
.
.
.
.
.
1/0/175
long
1/0/176
long
1/0/177
long
1/0/178
long
1/0/179
long
1/0/180
long
Table Continued
72
Connectivity and port setup (production network)
--More-or
(q)uit
1/1/1
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
1000
Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Down
Enable
Enable
Enable
10G Full
Up
Enable
Enable
long
1/1/2
long
1/1/3
long
1/1/4
long
1/1/5
long
1/1/6
long
1/1/7
long
1/1/8
long
1/1/9
long
1/1/10
long
1/1/11
long
1/1/12
long
1/1/13
long
1/1/14
long
1/1/15
long
1/1/16
10G
Full
long
Connectivity and port setup (production network)
73
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website.
Example: Show 1 GbE downlink ports: Moonshot-45Gc switch
The following example shows how to display the status of the 1 GbE downlink ports for the
Moonshot-45Gc switches. The status is displayed through the serial console port or the switch VSP.
<HP> display interface gigabitethernet brief
Brief information on interface(s) under bridge mode:
Link: ADM - administratively down; Stby - standby
Speed: (a)/A - auto
Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Interface
Link
Speed
Duplex
Type
PVID
GE1/0/1
UP
1G(a)
F(a)
A
1
GE1/0/2
DOWN
auto
A
A
1
GE1/0/3
DOWN
auto
A
A
1
GE1/0/4
DOWN
auto
A
A
1
GE1/0/5
DOWN
auto
A
A
1
GE1/0/6
DOWN
auto
A
A
1
GE1/0/7
DOWN
auto
A
A
1
GE1/0/8
DOWN
auto
A
A
1
GE1/0/9
DOWN
auto
A
A
1
GE1/0/10
DOWN
auto
A
A
1
GE1/0/11
UP
1G(a)
F(a)
A
1
GE1/0/12
DOWN
auto
A
A
1
GE1/0/13
DOWN
auto
A
A
1
GE1/0/14
DOWN
auto
A
A
1
GE1/0/15
DOWN
auto
A
A
1
GE1/0/16
DOWN
auto
A
A
1
GE1/0/17
DOWN
auto
A
A
1
GE1/0/18
UP
1G(a)
F(a)
A
1
GE1/0/19
DOWN
auto
A
A
1
GE1/0/20
UP
1G(a)
F(a)
A
1
GE1/0/21
DOWN
auto
A
A
1
GE1/0/22
DOWN
auto
A
A
1
GE1/0/23
DOWN
auto
A
A
1
GE1/0/24
DOWN
auto
A
A
1
Descriptio
n
Table Continued
74
Example: Show 1 GbE downlink ports: Moonshot-45Gc switch
GE1/0/25
DOWN
auto
A
A
1
GE1/0/26
DOWN
auto
A
A
1
GE1/0/27
DOWN
auto
A
A
1
GE1/0/28
DOWN
auto
A
A
1
GE1/0/29
DOWN
auto
A
A
1
GE1/0/30
DOWN
auto
A
A
1
GE1/0/31
DOWN
auto
A
A
1
GE1/0/32
DOWN
auto
A
A
1
GE1/0/33
UP
1G(a)
F(a)
A
1
GE1/0/34
DOWN
auto
A
A
1
GE1/0/35
DOWN
auto
A
A
1
GE1/0/36
DOWN
auto
A
A
1
GE1/0/37
DOWN
auto
A
A
1
GE1/0/38
DOWN
auto
A
A
1
GE1/0/39
DOWN
auto
A
A
1
GE1/0/40
DOWN
auto
A
A
1
GE1/0/41
DOWN
auto
A
A
1
GE1/0/42
DOWN
auto
A
A
1
GE1/0/43
DOWN
auto
A
A
1
GE1/0/44
DOWN
auto
A
A
1
GE1/0/45
UP
1G(a)
F(a)
A
1
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Show 10GbE downlink and uplink ports: Moonshot-45XGc switch
The following example shows how to display the status of the 10 GbE downlink ports for the
Moonshot-45XGc switch with a Moonshot-16SFP+ uplink module. This is displayed through the serial
console port or the switch VSP.
<HP> display interface ten-gigabitethernet brief
Brief information on interface(s) under bridge mode:
Link: ADM - administratively down; Stby - standby
Speed: (a)/A - auto
Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Interface
Link
Speed
Duplex
Type
PVID
XGE1/0/1
UP
1G(a)
F(a)
A
1
Descriptio
n
Table Continued
Example: Show 10GbE downlink and uplink ports: Moonshot-45XGc
switch 75
XGE1/0/2
DOWN
auto
A
A
1
XGE1/0/3
DOWN
auto
A
A
1
XGE1/0/4
DOWN
auto
A
A
1
XGE1/0/5
DOWN
auto
A
A
1
XGE1/0/6
DOWN
auto
A
A
1
XGE1/0/7
DOWN
auto
A
A
1
XGE1/0/8
DOWN
auto
A
A
1
XGE1/0/9
DOWN
auto
A
A
1
XGE1/0/10
DOWN
auto
A
A
1
XGE1/0/11
UP
1G(a)
F(a)
A
1
XGE1/0/12
DOWN
auto
A
A
1
XGE1/0/13
DOWN
auto
A
A
1
XGE1/0/14
DOWN
auto
A
A
1
XGE1/0/15
DOWN
auto
A
A
1
XGE1/0/16
DOWN
auto
A
A
1
XGE1/0/17
DOWN
auto
A
A
1
XGE1/0/18
UP
1G(a)
F(a)
A
1
XGE1/0/19
DOWN
auto
A
A
1
XGE1/0/20
UP
1G(a)
F(a)
A
1
XGE1/0/21
DOWN
auto
A
A
1
XGE1/0/22
DOWN
auto
A
A
1
XGE1/0/23
DOWN
auto
A
A
1
XGE1/0/24
DOWN
auto
A
A
1
XGE1/0/25
DOWN
auto
A
A
1
XGE1/0/26
DOWN
auto
A
A
1
XGE1/0/27
DOWN
auto
A
A
1
XGE1/0/28
DOWN
auto
A
A
1
XGE1/0/29
DOWN
auto
A
A
1
XGE1/0/30
DOWN
auto
A
A
1
XGE1/0/31
DOWN
auto
A
A
1
XGE1/0/32
DOWN
auto
A
A
1
XGE1/0/33
UP
1G(a)
F(a)
A
1
XGE1/0/34
DOWN
auto
A
A
1
XGE1/0/35
DOWN
auto
A
A
1
XGE1/0/36
DOWN
auto
A
A
1
XGE1/0/37
DOWN
auto
A
A
1
XGE1/0/38
DOWN
auto
A
A
1
Table Continued
76
Connectivity and port setup (production network)
XGE1/0/39
DOWN
auto
A
A
1
XGE1/0/40
DOWN
auto
A
A
1
XGE1/0/41
DOWN
auto
A
A
1
XGE1/0/42
DOWN
auto
A
A
1
XGE1/0/43
DOWN
auto
A
A
1
XGE1/0/44
DOWN
auto
A
A
1
XGE1/0/45
UP
1G(a)
F(a)
A
1
XGE1/1/1
UP
10G(a)
F(a)
A
1
XGE1/1/2
DOWN
auto
A
A
1
XGE1/1/3
DOWN
auto
A
A
1
XGE1/1/4
DOWN
auto
A
A
1
XGE1/1/5
DOWN
auto
A
A
1
XGE1/1/6
UP
10G(a)
F(a)
A
1
XGE1/1/7
DOWN
auto
A
A
1
XGE1/1/8
DOWN
auto
A
A
1
XGE1/1/9
UP
10G(a)
F(a)
A
1
XGE1/1/10
DOWN
auto
A
A
1
XGE1/1/11
DOWN
auto
A
A
1
XGE1/1/12
DOWN
auto
A
A
1
XGE1/1/13
DOWN
auto
A
A
1
XGE1/1/14
DOWN
auto
A
A
1
XGE1/1/15
DOWN
auto
A
A
1
XGE1/1/16
DOWN
auto
A
A
1
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Display 10GbE uplink ports: Moonshot-45Gc switch
The following example indicates how to display the status of the 10 GbE ports of the Moonshot-6SFP
uplink module from a Moonshot-45Gc switch. This is displayed through the serial console port or the
switch VSP.
<HP> display interface ten-gigabitethernet brief
Brief information on interfaces in bridge mode:
Link: ADM - administratively down; Stby - standby
Speed: (a) - auto
Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Example: Display 10GbE uplink ports: Moonshot-45Gc switch
77
Interface
Link
Speed
Duplex
Type
PVID
XGE1/1/1
DOWN
auto
A
A
1
XGE1/1/2
DOWN
auto
A
A
1
XGE1/1/3
DOWN
auto
A
A
1
XGE1/1/4
UP
10G(a)
F(a)
A
1
XGE1/1/5
DOWN
auto
A
A
1
XGE1/1/6
DOWN
auto
A
A
1
Descriptio
n
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Display 40 GbE uplink ports: Moonshot-45XGc/180XGc switch
The following example indicates how to display the status of the 40 GbE ports of the Moonshot-4QSFP+
uplink module from a Moonshot-45XGc/180XGc switch. This is displayed through the serial console port
or the switch VSP.
<HP> display interface fortygige brief
Brief information on interfaces in bridge mode:
Link: ADM - administratively down; Stby - standby
Speed: (a) - auto
Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Interface
Link
Speed
Duplex
Type
PVID
FGE1/1/1
UP
40G(A)
F(a)
A
1
FGE1/1/2
DOWN
auto
A
A
1
FGE1/1/3
DOWN
auto
A
A
1
FGE1/1/4
UP
auto
A
A
1
Descriptio
n
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Show 10 GbE downlink and uplink ports: Moonshot-180XGc
switch
The following example shows how to display the status of the 10 GbE downlink ports for the
Moonshot-180XGc switch with a Moonshot-16SFP+ uplink module. This is displayed through the serial
console port or the switch VSP.
<HP> display interface ten-gigabitethernet brief
Brief information on interface(s) under bridge mode:
Link: ADM - administratively down; Stby - standby
78
Example: Display 40 GbE uplink ports: Moonshot-45XGc/180XGc switch
Speed: (a)/A - auto
Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Interface
Link
Speed
Duplex
Type
PVID
XGE1/0/1:1 UP
1G(a)
F(a)
A
1
XGE1/0/1:2 UP
1G(a)
F(a)
A
1
XGE1/0/1:3 UP
1G(a)
F(a)
A
1
XGE1/0/1:4 UP
1G(a)
F(a)
A
1
XGE1/0/2:1 DOWN
auto
A
A
1
XGE1/0/2:2 DOWN
auto
A
A
1
XGE1/0/2:3 DOWN
auto
A
A
1
XGE1/0/2:4 DOWN
auto
A
A
1
XGE1/0/3:1 DOWN
auto
A
A
1
XGE1/0/3:2 DOWN
auto
A
A
1
XGE1/0/3:3 DOWN
auto
A
A
1
XGE1/0/3:4 DOWN
auto
A
A
1
XGE1/0/4:1 DOWN
auto
A
A
1
XGE1/0/4:2 DOWN
auto
A
A
1
XGE1/0/4:3 DOWN
auto
A
A
1
XGE1/0/4:4 DOWN
auto
A
A
1
XGE1/0/5:1 DOWN
auto
A
A
1
XGE1/0/5:2 DOWN
auto
A
A
1
XGE1/0/5:3 DOWN
auto
A
A
1
XGE1/0/5:4 DOWN
auto
A
A
1
XGE1/0/44: UP
1
1G(a)
F(a)
A
1
XGE1/0/44: DOWN
2
auto
A
A
1
XGE1/0/44: DOWN
3
auto
A
A
1
XGE1/0/44: DOWN
4
auto
A
A
1
XGE1/0/45: UP
1
1G(a)
F(a)
A
1
Descriptio
n
.
.
.
Table Continued
Connectivity and port setup (production network)
79
XGE1/0/45: DOWN
2
auto
A
A
1
XGE1/0/45: DOWN
3
auto
A
A
1
XGE1/0/45: DOWN
4
auto
A
A
1
XGE1/1/1
DOWN
auto
A
A
1
XGE1/1/2
DOWN
auto
A
A
1
XGE1/1/3
DOWN
auto
A
A
1
XGE1/1/4
DOWN
auto
A
A
1
XGE1/1/5
DOWN
auto
A
A
1
XGE1/1/6
DOWN
auto
A
A
1
XGE1/1/7
DOWN
auto
A
A
1
XGE1/1/8
DOWN
auto
A
A
1
XGE1/1/9
DOWN
auto
A
A
1
XGE1/1/10
DOWN
auto
A
A
1
XGE1/1/11
DOWN
auto
A
A
1
XGE1/1/12
DOWN
auto
A
A
1
XGE1/1/13
DOWN
auto
A
A
1
XGE1/1/14
DOWN
auto
A
A
1
XGE1/1/15
DOWN
auto
A
A
1
XGE1/1/16
DOWN
auto
A
A
1
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Troubleshooting port setup
The following information helps you avoid some of the most common errors in setting up the ports on
Moonshot.
80
•
Take care not to create network loops with the 4x10G splitter cable. See "Network loops."
•
When configuring QSFP+ uplink ports, a reboot of the switch is necessary.
•
Network traffic is disrupted when the switch is rebooted.
Troubleshooting port setup
Network loops
Network loop overview
Moonshot network switches are integrated into the chassis. This means you must take care when
connecting to environments or a network loop can result. This section helps you understand what a
network loop is and how to avoid undesired results with your Moonshot system. You can find videos
created for this section on the Hewlett Packard Enterprise website.
What is a network loop?
A network loop, or Layer 2 loop topology, is one in which multiple physical and logical paths exist between
two forwarding devices, such as an Ethernet switch. Such a topology exposes a condition where
broadcast traffic is multiplied and sent back and forth between switches over the redundant connections.
Switches cannot function in this environment because it creates an infinite feedback loop of broadcast
packets. Consider what happens when a microphone gets too close to an amplified speaker that it is
connected to. The microphone picks up the sound from the speaker and the speaker amplifies that signal
over and over again until all you hear is ear-splitting noise. This is analogous to what happens to a switch
when you put it into a loop topology. The effect is known as a broadcast storm.
When is a loop topology OK?
A loop topology is not always a bad thing. Most approaches to high availability networking involve having
identical redundant paths between two points on the network. As long as broadcast traffic is not allowed
to multiply over a loop topology, it is not harmful.
Methods for avoiding broadcast storms on a loop topology include the following:
•
Spanning Tree Protocol
•
TRILL
•
Link Aggregation
•
VLANs
STP and TRILL build a network topology representation that allows each switch to detect loops. In the
case of STP, some connections are disabled to break the loop topology. In the case of TRILL, all
connections remain active, but forwarding rules prevent broadcast traffic from following a loop path.
Link aggregation is a manual setting that informs the switch that multiple physical interfaces are to be
treated as a single logical interface. After multiple ports are known as a single logical interface, the switch
will not forward broadcast traffic through it more than once.
VLANs provide a way to mark traffic on the switch as belonging to a unique virtual network with its own
broadcast domain. Because broadcast traffic stays only within its L2 broadcast domain, there is no risk of
a broadcast storm if two redundant paths are part of different VLANs.
Uncontrolled loops - what happens?
What happens to a switch when a switch is exposed to an uncontrolled loop topology?
To understand the problem you must first understand what a network switch does. A simple switch has
three functions:
Network loops
81
1. Receive packets (ingress).
2. Analyze and route those packets to the appropriate interface.
3. Transmit those packets on that interface (egress).
The switch receives packets into its ingress pipeline then creates a slightly modified clone of that packet
and transmits that packet out its egress pipeline.
If the switch is exposed to a network loop, it routes those broadcast packets it recently generated back
into its uplink-facing ingress pipeline. It continues to do so until enough of these broadcast packets
regenerate and the switch bandwidth becomes saturated.
It is important to understand that this is a simplified example of what would occur during a single switch
exposed to a loop topology; as more switches are exposed to that same topology, the feedback effect
grows exponentially.
Situations to avoid in Moonshot configurations
This section provides the most common situations that can cause broadcast storms with a Moonshot
1500 system when STP or TRILL are disabled.
82
Situations to avoid in Moonshot configurations
Example: Multiple links
In this example, a Moonshot 1500 Chassis is installed with two Moonshot-4QSFP+ uplink modules
attached to an external ToR switch.
In this case a loopback has been created between uplink modules A and B because there is more than
one link between them.
Example: Redundant links
In this example, a Moonshot 1500 Chassis is installed with two Moonshot-6SFP+ uplink modules
attached to an external ToR switch.
In this case, a loopback has been created because both uplink modules A and B are uplinked to the
external switch as well to each other.
Example: Splitter cable
In this example, a Moonshot 1500 Chassis is installed with one Moonshot-4QSFP+ uplink module
attached to an external ToR switch through a QSFP+ to 4xSFP+ splitter cable.
In this case, the QSFP port is already converted into four separate interfaces. Therefore, a similar
topology to that of case #1 has been created; that is, there are multiple independent links between the
Moonshot switch and the external ToR switch.
Example: Multiple links
83
Preventing broadcast storms
The best way to ensure that network loops are prevented is to ensure that all of the switches on your
network have the same network loop management protocol enabled, such as STP or TRILL. Both of
these protocols automatically create a logical network that avoids topologies that can cause broadcast
storms.
Alternatively, you can use LAGs and/or VLANs to manually create a logical network that isolates the
broadcast domains.
Example: Preventing loops by creating a Link Aggregation Group
Using this example, you can create a LAG between Switches A and B. This will form a single interface,
and thus, a single broadcast domain between the switches instead of three independent interfaces.
For information on implementing LAGs on Moonshot-45G/180G and Moonshot-45Gc/45XGc/180XGc
switches, see "High availability networking."
Example: Preventing loops by creating VLANs to isolate broadcast
domains
In this example, both Switch A and Switch B are Moonshot-45G switches. The external switch is outside
the scope of this example.
By adding VLANs across each of the uplinks and relevant downlinks, you ensure that broadcast traffic will
not be forwarded back to the switch it originated from.
84
Preventing broadcast storms
Network loops
85
Network installation
Network installation overview
After your system is set up and links are verified using the Quick start section, you can review the
following examples of network installation. This is not an exhaustive set of examples, but is intended to
help you understand how to setup several select OS PXE environments. You can find videos created for
this section on the Hewlett Packard Enterprise website.
Performing a network installation of a Linux operating system to a Moonshot cartridge requires the use of
a PXE server. If you already have a PXE server on your network, go to "Modifying PXE for the
Moonshot serial console." If you do not have a PXE server, proceed to the next section, which provides
information on creating one.
Creating a PXE server
This section includes examples for each step in setting up a PXE server.
1. Basic network configuration
2. Configure DHCP service
3. TFTP service configuration
4. HTTP server for OS installation files
5. Prepare OS installation files for HTTP server
6. PXE boot files
Example: Basic network configuration
You must have a dedicated network interface that your server will use to provide DHCP and other
services to your installation targets. Take care when attaching this interface to a network. To avoid DHCP
addressing problems, avoid using a network on which another DHCP server is present.
For the purposes of this document, it is assumed that you have configured in the file /etc/sysconfig/
network-scripts/ifcfg-eth0 your first NIC (eth0) as follows:
DEVICE="eth0"
BOOTPROTO="static"
HWADDR="aa:bb:cc:dd:ee:ff"
NM_CONTROLLED="no"
ONBOOT="yes"
TYPE="Ethernet"
IPADDR="192.168.1.1"
NETMASK="255.255.255.0"
with
•
Static IP Address: 192.168.1.1
•
Network Mask: 255.255.255.0
The HWADDR value should match the MAC address of your NIC, or be absent from the configuration file.
86
Network installation
•
Disable the built-in firewall with the commands: chkconfig iptables off and service
iptables stop
•
Disable selinux with the command setenforce 0.
•
Edit /etc/sysconfig/selinux, changing the value of the SELINUX variable from enforcing to
disabled.
IMPORTANT:
These settings make it easier to set up your installation server, but also disable the firewall and
Security-Enhanced Linux (SELinux). Be sure that your installation server is running within a
private environment before making these changes..
For more information on on how to configure these services (rather than disabling them), have your
installation server accessible via the internet and see the Linux distribution documentation.
Example: Configure DHCP service
The purpose of DHCP is to provide IP addresses to your servers.
Install the ISC DHCP daemon with the command yum install dhcp.
Edit the file /etc/dhcp/dhcp.conf and replace the current contents with the following:
option domain-name "moonshotnet";
default-lease-time 600;
max-lease-time 7200;
authoritative;
allow booting;
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.150 192.168.1.249;
option bootfile-name "pxelinux.0";
next-server 192.168.1.1;
}
Edit /etc/sysconfig/dhcpd and set the value of DHCPARGS to eth0.
Run the commands chkconfig dhcpd on and service dhcpd start to start the DHCP service.
Example: TFTP service configuration
The TFTP service is used to provide the starting set of files (the Kernel and Ramdisk) to your network
booting system. The next-server parameter provided in the DHCP configuration specifies the IP
addresses of the TFTP server to access. The option bootfile-name parameter defines the name of
the boot file that the PXE client must request.
To install and configure the TFTP service on your installation server, complete the following steps:
1. Install the TFTP service with the command yum install tftp-server syslinux.
2. Enable and start the two required services with the following commands:
chkconfig tftp on
chkconfig xinetd on
service xinetd start
Example: Configure DHCP service
87
This creates and configures the TFTP server to use the directory.
3. /var/lib/tftpboot by default on your system to service TFTP requests. Populate the TFTP server
directory with the basic network boot files:
How to format this number?
cd /usr/share/syslinux
cp pxelinux.0 menu.c32 chain.c32 /var/lib/tftpboot
mkdir /var/lib/tftpboot/pxelinux.cfg
Example: HTTP server for OS installation files
HTTP server for OS installation files
While TFTP is used to provide the starting set of files, other protocols are used for the bulk of the files
used to deploy Linux. Most Linux distributions support installation using HTTP, NFS, and/FTP. The
following procedure installs and configures the Apache Web server so that it can provide the installation
files over HTTP.
1. Install Apache by entering the command: yum install httpd
2. Start Apache/httpd: with the command: chkconfig httpd on
3. Set the service to start automatically at boot time: service httpd start
By default, this configures a directory found at /var/www/html to provide files via your web server.
Example: Preparing OS installation files for HTTP server
Preparing OS installation files for HTTP server
The next task is to extract the operating system installation files from your Linux distribution installation
media. In most cases, you will be working with an ISO file containing the operating system to be installed.
Each model of HPE ProLiant server cartridges requires a specific minimum version of certain Linux
distributions, due to certain device and driver requirements. For specific details on which Linux
distribution versions are supported on the different ProLiant server cartridges, see the interactive OS
Support Matrix available on the Hewlett Packard Enterprise website (http://www.hpe.com/info/
ossupport).
The process for copying the installation files is essentially the same for all Linux distributions. The names
of the ISO files you have might vary from the names used in the following examples.
RHEL (and CentOS, Scientific Linux, Fedora)
1. Create a directory in your web server:
mkdir –p /var/www/html/os/rhel-6.5
2. Mount the ISO file with the command:
mount –o ro,loop RHEL-6.5Server-x86_64-DVD1.iso /mnt
3. Copy the contents of the ISO to the directory you created:
cp –a /mnt/* /var/www/html/os/rhel-6.5
4. Unmount the ISO:
88
Example: HTTP server for OS installation files
umount /mnt
Ubuntu
1. Create a directory in your web server:
mkdir –p /var/www/html/os/ubuntu-14.04
2. Mount the ISO file with the command:
mount –o ro,loop ubuntu-14.04-server-amd64.iso /mnt
3. Copy the contents of the ISO to the directory you created:
cp –a /mnt/* /var/www/html/os/ubuntu-14.04
4. Unmount the ISO:
umount /mnt
SLES
Certain ProLiant server cartridges require a special Kernel, Ramdisk, and/or packages for required device
drivers. SUSE provides the Kernel, Ramdisk, and/or packages using an image format called a kISO. You
must extract the contents of the kISO and the standard installation media.
If your Moonshot cartridge requires the use of a kISO with SLES11SP3, obtain the image from SUSE at
the SUSE website (http://drivers.suse.com/hp/HP-ProLiant-Moonshot/).
Extract the standard installation files from the media:
1. Create a directory in your web server:
mkdir –p /var/www/html/os/sles-11sp3
2. Mount the ISO file with the command:
mount –o ro,loop SLES-11-SP3-DVD-x86_64-GM.DVD1.iso /mnt
3. Copy the contents of the ISO to the directory you created:
cp –a /mnt/* /var/www/html/os/sles-11sp
4. Unmount the ISO: umount /mnt
Extract the kISO files:
5. Create a directory for the extracted files:
mkdir /var/www/html/os/m300-kiso-1.0
6. Mount the kISO:
mount –o ro,loop hp_proliant_m300-sles11sp3-x86-1.0.iso /mnt
7. Copy the files from the kISO to the directory you created:
cp –a /mnt/* /var/www/html/os/m300-kiso-1.0
8. Unmount the kISO: unmount /mnt
RHEL DUPs and SLES kISOs
Some updated drivers might have not been available in the main distributions at the time the hardware
was released. Some cartridge types require the use of an additional image to install such updated drivers.
RHEL calls these Driver Update Packs (or Process) and SUSE calls them kISOs. For information on
obtaining and using these files in a network installation, see the Operating System Deployment on HPE
Network installation
89
ProLiant Moonshot Server Cartridges User Guide on the Hewlett Packard Enterprise website (http://
www.hpe.com/support/hpesc).
Example: PXE boot files
After copying the OS installation files into your HTTP server, you must also copy the Kernel and Ramdisk
files to your TFTP server.
RHEL
1. Create a directory in your TFTP server:
mkdir –p /var/lib/tftpboot/rhel-6.5
2. Change directory to your web server:
cd/var/www/html/os/rhel-6.5/images/pxeboot
3. Copy the files:
cp initrd.img vmlinuz /var/lib/tftpboot/rhel-6.5
Ubuntu
1. Create a directory in your TFTP server:
mkdir –p /var/lib/tftpboot/ubuntu-14.04
2. Change directory to your web server:
cd /var/www/html/os/ubuntu-14.04/install/netboot/ubuntu-installer/amd64
3. Copy the files:
cp initrd.gz linux /var/lib/tftpboot/ubuntu-14.04
SLES
1. Create a directory in your TFTP server: mkdir –p /var/lib/tftpboot/sles-11-sp3
2. Change directory to your web server:cd /var/www/html/os/sles-11sp3/boot/x86_64/
loader
3. Copy the files:
cp initrd linux /var/lib/tftpboot/sles-11sp3
If your Moonshot cartridge requires a kISO, perform the following steps:
1. Make a directory for the files:
mkdir /var/lib/tftpboot/sles-11sp3/m300-kiso-1.0
2. Change directories:
cd /var/www/html/os/m300-kiso-1.0
3. Copy the files:
cp linux initrd /var/lib/tftpboot/sles-11sp3/m300-kiso-1.0
As indicated previously, if your cartridge requires the use of a Red Hat DUP, or SUSE kISO, see the OS
Deployment Guide. The guide has details on integrating it during the network installation.
90
Example: PXE boot files
Your file names might vary. The commands provided previously are just examples.
Modifying PXE for the Moonshot serial console
The following instructions provide the necessary steps to modify an existing PXE server for use with
Moonshot.
Example: Modifying your PXE configuration
Unlike traditional servers, Moonshot cartridges use a serial console to provide basic messages and
interaction with the operating system. The modifications to your PXE configuration are made in two
places in the pxelinux.cfg/default file.
At the top of the file, add:
SERIAL 0 9600
CONSOLE 0
These statements cause the PXE environment itself to use the serial console on the cartridge. Secondly,
for each bootable entry (see following examples) in your pxelinux.cfg/default file, you must add:
console=ttyS0,9600n8
to the "append" section of each entry. For instance, an entry for Red Hat Enterprise Linux (RHEL) 6.5
which originally appeared as:
LABEL rhel-6.5
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img
would be modified to:
LABEL rhel-6.5
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img console=ttyS0,9600n8
The following shows a complete example pxelinux.cfg/default file that contains a bootable
installation entry for all three of the major Linux distributions:
SERIAL 0 9600
CONSOLE 0
PROMPT 0
ONTIMEOUT local
TIMEOUT 300
DEFAULT menu.c32
LABEL local
kernel chain.c32
append hd0 0
LABEL rhel-6.5
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img repo=http://192.168.1.1/os/rhel-6.5
console=ttyS0,9600n8
LABEL ubuntu-14.04
kernel ubuntu-14.04/linux
append initrd=ubuntu-14.04/initrd.gz console=ttyS0,9600n8
LABEL sles-11sp3
kernel sles-11sp3/m300-kiso-1.0/linux
append initrd=sles-11sp3/m300-kiso-1.0/initrd
install=http://192.168.1.1/os/sles-11sp3
addon=http://192.168.1.1/os/m300-kiso-1.0 console=ttyS0,9600n8
Modifying PXE for the Moonshot serial console
91
The append lines included earlier occur on a single line and do not "wrap" as they are shown here. The
last entry, for SLES 11 SP3, also shows the use of a kISO from SUSE to provide additional network
drivers. See the documentation and OS support matrix for your selected cartridge to determine if a kISO
or a DUP for RHEL is required.
Adding the PXE boot files from your Linux distributions
The following example configuration will instruct your cartridge, when network booted (described in the
following steps), to load a Kernel and Ramdisk used to install Linux. Make these files available on your
PXE server. The following examples use three subdirectories within your PXE/TFTP server named
rhel-6.5, Ubuntu-14.04, and sles-11sp3. Create the directories at the top level of your PXE/TFTP server
(same directory that the pxelinux.cfg directory exists) then copy these files from the ISO file used by your
Linux distribution. Download the ISO files from your Linux distribution vendor, or reseller.
For RHEL-6.5, mount the ISO using the following command to mount the contents to a directory
named /mnt on your system:
mount –o ro,loop RHEL6.5-Server-x86_64-DVD1.iso /mnt
Then, copy the files named /mnt/images/pxeboot/initrd.img and /mnt/images/pxeboot/
vmlinuz to the rhel-6.5 directory created previously.
Unmount the ISO file using the following command:
umount /mnt
Mount the SLES ISO files and copy the required files:
/mnt/boot/x86_64/loader/linux and /mnt/boot/x86_64/initrd
Mount the Ubuntu ISO files and copy the required files:
/mnt/install/netboot/ubuntu-installer/amd64/linux and /mnt/install/netboot/ubuntu-installer/amd64/
initrd.gz
Configuring the server
Like traditional servers, Moonshot cartridges and nodes support the concept of boot device order.
There are fewer devices to choose from in Moonshot, but you are still required to specify which device
you want, and when.
There are two commands to manipulate the boot order of your nodes:
•
Permanent Boot Order: set node boot device c1n1
•
One Time Boot Order: set node bootonce device c1n1
Both of these commands modify the boot order of the first node (n1) of the first cartridge (c1). Valid values
of the device field are pxe, hdd, and m.2. m.2 is available only on certain cartridges that support the flash
storage.
•
To set your node to boot from the network a single time, use the following command:
set node bootonce pxe c1n1
•
To set the same node to boot from the network at ALL times, use the following command:
set node boot pxe c1n1
•
92
To make the same node boot from a hard drive, use the same commands, but use hdd as the device
parameter instead.
Adding the PXE boot files from your Linux distributions
With the boot device configuration set to boot from the network, power on your node using the command:
set node power on c1n1
If the cartridge was already powered on, power it off with the following command:
set node power off force c1n1
Then, issue the power-on command.
In the same (or a separate) SSH connection to your Chassis Manager, open a virtual serial port session
(as described in the Network management on page 29 section) to interact and complete the installation
using the following command:
connect node vsp c1n1
With your PXE configuration set as described previously, and your node configured to boot from the
network, you will be presented with a menu and allowed to choose to either boot from the local hard drive,
or to install one of the chosen Linux distributions. When the system boots, it loads the interactive installer
used by your distribution.
Automating your Linux installation
Each of the Linux distributions supports a mechanism to automate the process of deploying Linux to a
server. These mechanisms require the use of an additional configuration file. For RHEL, the file is called a
Kickstart file, for SLES it is an AutoYaST file, and Ubuntu names it a preseed file. Regardless of what it is
called, it is a file, made available typically on a Web or ftp server, that contains the configurations
necessary to "answer" the questions you usually answer manually during an interactive installation.
Example: Kickstart file for RHEL
text
url --url=http://192.168.1.1/os/rhel-6.5/
lang en_US.UTF-8
keyboard us
network --device eth0 --noipv6 --bootproto dhcp
network --device eth1 --noipv6 --bootproto dhcp
rootpw --plaintext moonshot
user --name="moonshot" --gecos="moonshot" --shell="/bin/bash" --plaintext -password=moonshot
Automating your Linux installation
93
authconfig --enableshadow --passalgo=sha512 --enablefingerprint
firewall --disabled
selinux --disabled
timezone America/Chicago
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto
console=ttyS0n8"
poweroff
#reboot
zerombr
clearpart --all --drives=sda
part /boot --fstype=ext4 --size=700
part swap --recommended
part / --fstype=ext4 --size=1 --grow
%post
cat >>/etc/sudoers <<EOF
moonshot ALL=(ALL) ALL
EOF
%end
%packages
@base
@core
%end
Automated installations also typically require that the entire contents of the ISO file provided be extracted
and made similarly available on a Web or FTP server. The configuration above instructs the installer to
obtain the installation files from a Web server available to the server at http://192.168.1.1/os/rhel-6.5/.
For more information on extracting the installation files and storing them on your Web server, see the
Operating System Deployment on HPE ProLiant Moonshot Server Cartridges User Guide starting
on page 46. The same guide also provides an example AutoYaST file (Page 70) for SUSE, and preseed
file (Page 52) for Ubuntu. They are both somewhat longer than the format used by the Kickstart file, so
they are not included here.
Example: Modifying PXE configuration file for an RHEL kickstart
To instruct your network installation to use these installation configuration files, you must update your
pxelinux.cfg/default file in your PXE server to include additional parameters for each installation
that will instruct it to load and use the corresponding file.
For example, the above RHEL PXE configuration file would change the original entry from:
LABEL rhel-6.5
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img repo=http://192.168.1.1/os/rhel-6.5
console=ttyS0,9600n8
To:
LABEL rhel-6.5-automatic
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img ksdevice=auto ks=http://192.168.1.1/answers/rhel-6.5.cfg console=ttyS0,9600n8
94
Example: Modifying PXE configuration file for an RHEL kickstart
The append lines are a single contiguous line without breaks. This modification instructs the installer to
obtain the installation configuration file (Kickstart, in this case) from a Web server at 192.168.1.1, in a
subdirectory called answers. The configuration file itself is named rhel-6.5.cfg. You can name these
whatever you want, but the boot entries must be configured to match what you choose.
A complete example pxelinux.cfg/default file that shows both interactive and automatic examples
of each of the main operating systems is:
SERIAL 0 9600
CONSOLE 0
PROMPT 0
ONTIMEOUT local
TIMEOUT 300
DEFAULT menu.c32
LABEL local
kernel chain.c32
append hd0 0
LABEL rhel-6.5-auto
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img ksdevice=eth0 ks=http://192.168.1.1/answers/rhel-6.5.cfg console=ttyS0,9600n8
LABEL rhel-6.5-interactive
kernel rhel-6.5/vmlinuz
append initrd=rhel-6.5/initrd.img repo=http://192.168.1.1/os/rhel-6.5 console=ttyS0,9600n8
LABEL ubuntu-14.04-auto
kernel ubuntu-14.04/linux
append initrd=ubuntu-14.04/initrd.gz priority=critical auto=true
preseed/url=http://16.84.217.164/answers/ubuntu-14.04.cfg interface=em2 console=ttyS0,9600n8
LABEL ubuntu-14.04-interactive
kernel ubuntu-14.04/linux
append initrd=ubuntu-14.04/initrd.gz console=ttyS0,9600n8
LABEL sles-11sp3-auto
kernel sles-11sp3/m300-kiso-1.0/linux
append initrd=sles-11sp3/m300-kiso-1.0/initrd
install=http://192.168.1.1/os/sles-11sp3
addon=http://192.168.1.1/os/m300-kiso-1.0
autoyast=http://192.168.1.1/answers/sles-11sp3.xml console=ttyS0,9600n8
LABEL sles-11sp3-interactive
kernel sles-11sp3/m300-kiso-1.0/linux
append initrd=sles-11sp3/m300-kiso-1.0/initrd
install=http://192.168.1.1/os/sles-11sp3
addon=http://192.168.1.1/os/m300-kiso-1.0 console=ttyS0,9600n8
If you execute an automated installation, Linux is deployed to your system without further interaction from
you. You can customize your installation by manipulating the parameters included in the installation
configuration file.
Troubleshooting network installation
The following information helps you avoid some of the most common errors in network installation on
Moonshot.
•
If you do not see any output on your screen as your node PXE boots, make sure that the "SERIAL 0
9600" and "CONSOLE 0" parameters are specified at the top of the pxelinux.cfg/default
configuration file on your PXE/TFTP server.
•
If you do not see any output on your screen after you choose a boot entry (configured as above) verify
that the "append" line includes the parameter "console=ttyS0,9600n8".
•
If your cartridge node attempts to PXE, but does not get a DHCP assigned IP address, verify that your
network has DHCP enabled by connecting a laptop and then running "ipconfig" in a command
window (or "ifconfig" on Linux) to verify that you can obtain an IP address dynamically.
Troubleshooting network installation
95
96
•
If your cartridge node attempts to PXE, obtains an IP address, but then cannot obtain a PXE
configuration file (default), verify that your PXE server is providing TFTP access to the file
"pxelinux.0". On another system with the tftp client installed, run "tftp
your.server.ip.address" (replacing with your PXE server IP) then "get pxelinux.0" to verify
that TFTP is working.
•
If your cartridge node PXE’s successfully, but then encounters a problem loading your kernel and/or
ramdisk, ensure that the files are on your TFTP server and in the directory indicated in the
configuration file (pxelinux.cfg/default), and that they have adequate permissions to be read
over the network (0644). Also ensure that any network installation files and configuration files are
available at their specified URL addresses.
•
If you try to start a service and you get a [Failed], you can recall the last 50 executed lines to find
the error by doing the following commands: cd /var/log and tail -50 messages |more
Network installation
Spanning Tree Protocol
Spanning Tree Protocol overview
This advanced topic for network use is a method to prevent network loops. With your Moonshot system
connected and linked, this section helps you understand how you might use this feature. You can find
videos created for this section on the Hewlett Packard Enterprise website.
What is Spanning Tree Protocol?
Spanning Tree Protocol is a layer 2 protocol that provides a tree topology for switches on a bridged LAN.
STP allows a network to have redundant paths without the risk of network loops. STP uses the spanningtree algorithm to provide a single path between end stations on a network.
Moonshot switches support STP, RSTP, and RSTP.
How STP works
The switches (bridges) that participate in the spanning tree elect a switch to be the root bridge for the
spanning tree. The root bridge is the switch with the lowest bridge ID, which is computed from the unique
identifier of the bridge and its configurable priority number. When two switches have an equal bridge ID
value, the switch with the lowest MAC address is the root bridge.
After the root bridge is elected, each switch finds the lowest-cost path to the root bridge. The port that
connects the switch to the lowest-cost path is the root port on the switch. The switches in the spanning
tree also determine which ports have the lowest-path cost for each segment. These ports are the
designated ports.
Only the root ports and designated ports are placed in a forwarding state to send and receive traffic. All
other ports are put into a blocked state to prevent redundant paths that might cause loops.
To determine the root path costs and maintain topology information, switches that participate in the
spanning tree use BPDUs to exchange information.
Types of Spanning Tree Protocols
Spanning Tree Protocol (STP)
Spanning Tree Protocol (IEEE 802.1D) is a standard requirement of Layer 2 switches that allows bridges
to automatically prevent and resolve L2 forwarding loops.
Rapid Spanning Tree Protocol (RSTP)
Rapid Spanning Tree Protocol detects and uses network topologies to enable faster spanning tree
convergence after a topology change, without creating forwarding loops. RSTP is considerably faster than
STP.
Multiple Spanning Tree Protocol (MSTP)
Multiple Spanning Tree operation maps VLANs to spanning tree instances. Packets assigned to various
VLANs are transmitted along different paths within MSTP Regions (MST Regions). Regions are one or
more interconnected MSTP bridges with identical MSTP settings. The MSTP standard lets administrators
assign VLAN traffic to unique paths.
Per-VLAN Spanning Tree (PVST)
Per-VLAN Spanning Tree operation allows every VLAN configured in the network to have its own
spanning tree, which increases utilization of links and bandwidth. Because each VLAN runs STP or RSTP
independently, a spanning tree only serves its own VLAN.
Spanning Tree Protocol
97
Setting Spanning Tree mode
Example – Set STP mode: Moonshot-45G/180G
Use the following information to set up Moonshot-45G/180G Switches with different types of spanning
trees.
Set STP Mode
In configuration mode, you can set the spanning tree mode by running the following commands:
•
STP: spanning-tree forceversion 802.1d
•
RSTP: spanning-tree forceversion 802.1w
•
MSTP: spanning-tree forceversion 802.1s
Change STP Device Priority
In configuration mode, you can set the spanning tree device priority:
•
STP/RSTP: spanning-tree mst priority 0 <priority>
•
MSTP: spanning-tree mst priority <instance-id> <priority>
Enable/Disable STP
By default, STP is enabled on the Moonshot-45G/180G switch. In configuration mode, you can:
•
Disable STP with the following command:
no spanning-tree
•
Enable STP using the following command:
spanning-tree
•
Save your configuration using the following command:
(Routing) # write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example – Set STP mode: Moonshot-45Gc/45XGc/180XGc
Use the following information to set up Moonshot-45Gc/45XGc/180XGc Switches with different types of
spanning trees.
Set STP Mode
In system-view, you can set the spanning tree mode by running the following command:
stp mode { mstp | pvst | rstp | stp }
Change STP Device Priority
In system-view, you can set the spanning tree device priority by running the following command:
98
Setting Spanning Tree mode
stp priority {1-32768}
Enable/Disable STP
By default, STP is enabled on the Moonshot-45Gc/45XGc/180XGc switch. In system view, you can:
Disable STP with the following command:
undo stp global enable
Enable STP using the following command:
stp global enable
Save your configuration using the following command:
<HP>save
The current configuration will be written to the device. Are you sure? [Y/N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Troubleshooting STP setup
The main issue encountered with STP setup is not having an STP compatible network; for STP to work,
all switches on the network must have STP enabled. When a switch with STP enabled is linked to a
switch that does not, that link is shut down.
Troubleshooting STP setup
99
VLANs
VLAN overview
The VLAN standard, IEEE 802.1Q, is a standard that defines a system for tagging Ethernet frames and
procedures for handling such frames. The Moonshot Switch Module software is in full compliance with
IEEE 802.1Q VLAN tagging. Following are two simple examples of VLAN configuration. For more
information, see the switch documentation.
Isolating cartridges to Network A versus Network B
Isolate a cartridge downlink interface to Network A (uplink 1/1/1) while having another cartridge network
interface isolated to Network B (uplink 1/1/2).
Step 1—Packets come into the switch.
Step 2—Packets get tagged in ingress pipeline.
Step 3—Tagged packets move to egress pipeline.
100
VLANs
Step 4—Packets are untagged and forwarded to appropriate network.
Example: VLAN - untagged in, untagged out, C1 to Network A switch, C2 to
Network B switch (Moonshot-45G/180G)
Isolate some servers onto Network A (uplink 1/1/1) while having another cartridge network interface
isolated to Network B (uplink 1/1/2).
(Routing) >enable
# create the private VLANs that are used to separate Network A and Network B traffic
(Routing) #vlan database
(Routing) (Vlan)#vlan 450
(Routing) (Vlan)#vlan 550
(Routing) (Vlan)#exit
(Routing) #config
(Routing) (config)#interface 1/1/1! Setup Network A uplink on private vlan
450
(Routing) (interface 1/1/1)#vlan pvid 450! tells the switch to tag all
frames ingressing on 1/1/1 with 450
(Routing) (interface 1/1/1)#vlan participation include 450
(Routing) (interface 1/1/1)#exit
(Routing) (config)#interface 1/1/2! Setup Network B uplink on private vlan
550
(Routing) (interface 1/1/2)#vlan pvid 550! tells the switch to tag all
frames ingressing on 1/1/2 with 550
(Routing) (interface 1/1/2)#vlan participation include 550
(Routing) (interface 1/1/2)#exit
Example: VLAN - untagged in, untagged out, C1 to Network A switch, C2 to
Network B switch (Moonshot-45G/180G) 101
(Routing) (config)#interface 1/0/1! Use these steps for a cartridge that you
want on Network A
(Routing) (interface 1/0/1)#vlan pvid 450! tells the switch to tag all
frames ingressing on 1/0/1 with 450
(Routing) (interface 1/0/1)#vlan participation include 450! tells switch
that this port can pass or receive traffic if the VLAN is 450
(Routing) (interface 1/0/1)#vlan participation exclude 1! Do not let vlan 1
traffic participate on this port
(Routing) (interface 1/0/1)#exit
(Routing) (config)#interface 1/0/2! Use these steps for a cartridge that you
want on Network B
(Routing) (interface 1/0/2)#vlan pvid 550! tells the switch to tag all
frames ingressing on 1/0/2 with 550
(Routing) (interface 1/0/2)#vlan participation include 550! tells switch
that this port can pass or receive traffic if the VLAN is 550
(Routing) (interface 1/0/2)#vlan participation exclude 1! don't let vlan 1
traffic participate on this port
(Routing) (interface 1/0/2)#exit
(Routing) (config)#exit
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n)y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: VLAN - untagged in, untagged out, C1 to Network A switch, C2 to
Network B switch (Moonshot-45Gc/45XGc/180XGc)
Isolate some servers onto Network A (uplink 1/1/1) while having another cartridge network interface
isolated to Network B (uplink 1/1/2).
# create two vlans 450 (Network A) and 550 (Network B)
<HP>system-view
System View: return to User View with Ctrl+Z.
[HP]vlan 450
[HP-vlan450]vlan 550
[HP-vlan550]quit
# Set up Network A uplink on private vlan 450 as untagged on interface forty1/1/1
[HP]interface FortyGigE 1/1/1
[HP-FortyGigE1/1/1]port access vlan 450
[HP-FortyGigE1/1/1]quit
# Set up Network B uplink on private vlan 550 as tagged on interface forty1/1/2 and prevent vlan 1 traffic
from passing
[HP]interface FortyGigE 1/1/2
[HP-FortyGigE1/1/2]port access vlan 550
102 Example: VLAN - untagged in, untagged out, C1 to Network A switch,
C2 to Network B switch (Moonshot-45Gc/45XGc/180XGc)
[HP-FortyGigE1/1/2]quit
# configure downlink (server) ports as untagged and vlan 450 (network A) and 550 (network B)
[HP]int ten 1/0/1
[HP-Ten-GigabitEthernet1/0/1]port access vlan 450
[HP-Ten-GigabitEthernet1/0/1]quit
[HP]int ten 1/0/2
[HP-Ten-GigabitEthernet1/0/2]port access vlan 550
[HP-Ten-GigabitEthernet1/0/2]quit
<HP>save
The current configuration will be written to the device. Are you sure? [Y/
N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference
and the Moonshot-45XGc Switch Layer 3 – IP Routing Configuration Guide on the Hewlett Packard
Enterprise website (http://www.hpe.com/info/moonshot/networking.
No tagging
Have the network port of some servers VLAN tagged on Network A (uplink 1/1/1) while having the
network interface of another cartridge to Network B (uplink 1/1/2) with no tagging.
Step 1—Packet comes into switch untagged.
Step 2—Packet gets tagged in ingress pipeline.
No tagging
103
Step 3—Tagged packets out egress pipeline.
Example: VLAN - untagged in, tagged out, C1 to Network A (Moonshot-45G/
180G)
Have the network port of some servers VLAN tagged on Network A (uplink 1/1/1) while having the
network port on other servers on Network B (uplink 1/1/2) with no tagging.
(Routing) >enable
# create the vlan
(Routing) #vlan database
(Routing) (Vlan)#vlan 450
(Routing) #exit
(Routing) #config
(Routing) (config)#interface 1/0/1
(Routing) (Interface 1/0/1)#vlan pvid 450 # tells the switch to tag all
frames ingressing on 1/0/1 with 450
(Routing) (Interface 1/0/1)#vlan participation include 450 # tells switch
that this port can pass or receive traffic if the VLAN is 450
(Routing) (Interface 1/0/1)#vlan participation exclude 1 # Do not let vlan 1
traffic participate on this port
(Routing) (Interface 1/0/1)#exit
(Routing) (config)#interface 1/1/1
(Routing) (Interface 1/1/1)#vlan participation include 450
(Routing) (Interface 1/1/1)#vlan tagging 450 # on egress, continue to tag –
otherwise it would have stripped the tag
(Routing) (Interface 1/1/1)#vlan participation exclude 1# Do not let vlan 1
104 Example: VLAN - untagged in, tagged out, C1 to Network A
(Moonshot-45G/180G)
traffic participate on this port
(Routing) (Interface 1/1/1)#exit
(Routing) (config)#exit
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: VLAN - untagged in, tagged out, C1 to Network A
(Moonshot-45Gc/45XGc/180XGc)
Have the network port of some servers VLAN tagged on Network A (uplink 1/1/1) while having the
network interface of another cartridge to Network B (uplink 1/1/2) with no tagging.
# create vlan 450 (Network A)
<HP>system-view
System View: return to User View with Ctrl+Z.
[HP]vlan 450
[HP-vlan450]quit
# Set up Network A uplink on private vlan 450 as tagged on interface forty1/1/1 and prevent vlan 1 traffic
from passing
[HP]interface FortyGigE 1/1/1
[HP-FortyGigE1/1/1]port link-type trunk
[HP-FortyGigE1/1/1]port trunk permit vlan 450
[HP-FortyGigE1/1/1]undo port trunk permit vlan 1
[HP-FortyGigE1/1/1]quit
# Configure downlink (server) ports as untagged and vlan 450 (network A)
[HP]int ten 1/0/1
[HP-Ten-GigabitEthernet1/0/1]port access vlan 450
[HP-Ten-GigabitEthernet1/0/1]quit
<HP>save
The current configuration will be written to the device. Are you sure? [Y/N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
Example: VLAN - untagged in, tagged out, C1 to Network A (Moonshot-45Gc/
45XGc/180XGc) 105
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference
and the Moonshot-45XGc Switch Layer 3 – IP Routing Configuration Guide on the Hewlett Packard
Enterprise website (http://www.hpe.com/info/moonshot/networking).
106
VLANs
Port mirroring on a Moonshot switch
Port monitoring overview
After your Moonshot system is connected and linked, this section helps you understand how you can set
up and disable port mirroring in Moonshot. You can find videos created for this section on the Hewlett
Packard Enterprise website.
Port mirroring is the process of copying the packets passing through a port to the monitoring port
connecting to a monitoring device for packet analysis. Network engineers or administrators use port
mirroring to analyze and debug data or diagnose errors on a network. It helps administrators keep a close
eye on network performance and alerts them when problems occur. It can be used to mirror either
inbound or outbound traffic (or both) on single or multiple interfaces.
This guide provides information for configuring port mirroring via the switch command line interface. It
shows the steps required to setup port mirroring, and the necessary steps required to setup the mirroring
if a network breakout board is used in the port mirroring configuration.
Setting up port mirroring
Setting up port mirroring is slightly different depending on the switch installed in the system. This section
provides examples on how to set up port mirroring depending on the switches installed in the Moonshot
System.
Example: Set up port mirroring on Moonshot-45G/180G switches
Set up port mirroring where port 1/1/1 will receive all network traffic on port 1/0/1.
# Log into switch
User:Admin
Password: # press ‘Enter’ as there is no password
(Routing) >enable
(Routing) #configure
# Set up the port 1/0/1 as the mirrored port.
(Routing) (Config)#monitor session 1 source interface 1/0/1
# Set up the port 1/1/1 (probe port) to watch the mirrored port.
(Routing) (Config)#monitor session 1 destination interface 1/1/1
# Enables port mirroring.
(Routing) (Config)#monitor session 1 mode
(Routing) (Config)#exit
# Display the state of the mirrored port.
(Routing) #show monitor session 1
# Lists all the ports and their current speed/status
(Routing) #show port all
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Port mirroring on a Moonshot switch
107
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Set up local port mirroring on Moonshot-45XGc/180XGc switches
Set up port mirroring where port 1/1/1 will receive all network traffic on port 1/0/3 and 1/0/4.
# Create local mirroring group 1.
<HP> system-view
[HP] mirroring-group 1 local
# Configure Ten-GigabitEthernet 1/0/3 and Ten-GigabitEthernet 1/0/4 as source ports, and port FortyGigE
1/1/1 as the monitor port for local mirroring group 1. Note: only FortyGigE 1/1/1 port can be used for a
monitoring port.
[HP] mirroring-group 1 mirroring-port ten-gigabitethernet 1/0/3 tengigabitethernet 1/0/4 both
[HP] mirroring-group 1 monitor-port fortygige 1/1/1
# Disable the spanning tree feature on the monitor port FortyGigE 1/1/1.
[HP] interface fortygige 1/1/1
[HP-FortyGigE1/1/1] undo stp enable
[HP-FortyGigE1/1/1] quit
[HP]
# Display information about all mirroring groups.
[HP] display mirroring-group all
Mirroring group 1:
Type: Local
Status: Active
Mirroring port:
Ten-GigabitEthernet1/0/3 Both
Ten-GigabitEthernet1/0/4 Both
Monitor port: FortyGigE 1/1/1
<HP>save
The current configuration will be written to the device. Are you sure? [Y/N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Network Management and Monitoring
Configuration Guide on the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/
networking).
108 Example: Set up local port mirroring on Moonshot-45XGc/180XGc
switches
Removing port mirroring
Removing port mirroring is slightly different depending on the switch installed in the system. This section
provides examples on how to remove port mirroring depending on the switches installed in the Moonshot
System.
Example: Removing port mirroring on Moonshot-45G/180G switches
After port mirroring is no longer required, it can be removed as shown in the following example.
#Log into switch vsp session.
hpiLO-> connect switch vsp [<sa> / <sb>]
User:Admin
Password: // press ‘Enter’ as there is no password
(Routing) >enable
(Routing) #configure
#If port mirroring is already enabled, disable it.
(Routing) (Config)#monitor session 1 mode
#Disable the mirroring port.
(Routing) (Config)#no monitor session 1 source interface 1/0/<port-num>
#Remove the probe port.
(Routing) (Config)#no monitor session 1 destination interface
(Routing) (Config)#exit
#Verifies port mirroring has been disabled and mirror and probe ports have been removed.
(Routing) #show monitor session 1
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Removing port mirroring on Moonshot-45XGc/180XGc switches
After port mirroring is no longer required, it can be removed as shown in the following example.
#Remove local mirroring group 1.
<HP> system-view
[HP] undo mirroring-group 1
# Enable the spanning tree feature on port FortyGigE 1/1/1.
[HP] interface fortygige 1/1/1
Removing port mirroring
109
[HP-FortyGigE1/1/1] stp enable
[HP-FortyGigE1/1/1] quit
[HP]
# Display information about all mirroring groups.
[HP] display mirroring-group all
<HP>save
The current configuration will be written to the device. Are you sure? [Y/
N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Network Management and Monitoring
Configuration Guide on the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/
networking).
110
Port mirroring on a Moonshot switch
High availability networking
High availability networking overview
After your Moonshot system is connected and linked, this section helps you understand certain steps to
configure your system for high availability. This is not an exhaustive set of examples but is intended to
help you understand and get started on setting up your switches and then your downstream servers for
high availability. You can find videos created for this section on the Hewlett Packard Enterprise website.
This section is divided into two main areas of increased bandwidth and/or availability of networking within
a Moonshot chassis. The first area covered is on switch stacking. This is followed by LAGs, which can be
formed using the switch uplink ports and can be formed by server cartridge NICs and the switch downlink
ports.
Switch stacking and IRF fabrics
This section explains how to connect two or more Moonshot switches together in a stack. In a stacked
configuration, a group of switches can be managed as if they were a single switch that has the port
capacity of the sum of all the ports of these combined switches, minus the stacking ports.
•
MSTP + VRRP makes the topology complicated and converges slowly.
•
Stacking/IRF allows customers to connect multiple devices through physical stacking/IRF ports to
combine them into a logical device.
The examples below show a simple stacking configuration of two Moonshot switches (uplink modules
shown). The blue and green lines represent the cables used to create the stacking links. In these
examples, the second cable provides stack resiliency, so that if one of the stacking links goes down, the
switches remain stacked together.
High availability networking
111
Example: Stacking the Moonshot-45G/180G switch modules
Procedure
1. Make sure that all switches have the latest software image.
2. Identify one of the switches to be the manager and give it the highest priority value. By default, all
switches are configured with a priority of 1, the lowest value.
(Routing) #config
(Routing) (Config)#switch 1 priority 15
The priority of the second switch can be left at 1 which is the default setting.
3. On each stack member, configure the Ethernet ports that are to be used as stacking ports. See the
basic section in the beginning of this document for a list of available ports that can be used as stacking
ports. Use the following steps to configure the two stacking ports shown in the previous example. The
switch must be reset (with the reload command) for the changes to take effect.
(Routing) #config
(Routing) (Config)#stack
(Routing) (Config-stack)#stack-port 1/1/1 stack
(Routing) (Config-stack)#stack-port 1/1/2 stack
(Routing) (Config-stack)#exit
(Routing) (Config)#exit
(Routing) #reload
Are you sure you would like to reset the system? (y/n)y
4. Monitor the console port of the switch with the highest priority, also known as the manager switch.
5. Connect the uplink ports of both switches, as shown in the previous example. The lower priority switch
will reboot to become a part of the stack.
6. To verify whether the other switch has joined the stack, monitor the manager switch using the show
switch command as shown in the following example:
(Routing) #show switch
112
Example: Stacking the Moonshot-45G/180G switch modules
SW
Management
Switch
Standby
Status
Preconfig
Model ID
Plugged-in
Model ID
---
----------- ---------
----------- ----------- --------------------
Mgmt Sw
Moonshot-18 Moonshot-18 OK
0G
0G
----------1
2.0.0.16
2
Stack Mbr
Oper Stby
Moonshot-18 Moonshot-18 OK
0G
0G
2.0.0.1
7. Save your configuration.
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Creating a stacking or Intelligent Resilient Framework fabric with
multiple Moonshot-45XGc/180XGc switch modules
Procedure
1. Make sure that all switches have the latest software image.
2. Number the switches in the IRF fabric.
The switch chosen to be the manager or master will have number 1. Since the default for all switches
is 1, the manager switch does not need to be renumbered. Renumber the second switch as 2:
<HP>system-view
[HP]irf member 1 renumber 2
Example: Creating a stacking or Intelligent Resilient Framework fabric with
multiple Moonshot-45XGc/180XGc switch modules 113
[HP]save
[HP]quit
<HP>reboot
3. Change the priority of the master switch to the highest priority of 32.
The priority of the other IRF members can stay at 1, the default. Use the following instructions:
<HP>system-view
[HP]irf member 1 priority 32
[HP]save
4. On each IRF member, configure the IRF ports that are to be used to connect the two IRF members
together forming an IRF fabric.
There are three basic steps to create the IRF ports:
a. Shut down the uplink ports.
b. Bond uplink Ethernet ports to a specific IRF port.
c. Turn on the uplink ports.
Use the following instructions to create two IRF ports on switch 1 using two ports from a
Moonshot-4QSFP+ Uplink Module.
NOTE:
If a single 40G port is sufficient for the IRF connection, configure only one IRF port.
<HP>system-view
[HP]interface fortygige 1/1/1
[HP-FortyGigE1/1/1]shutdown
[HP]interface fortygige 1/1/2
[HP-FortyGigE1/1/2]shutdown
[HP]quit
[HP]irf-port 1/1
[HP-irf-port1/1]port group interface fortygige 1/1/1
[HP-irf-port1/1]quit
[HP]irf-port 1/2
[HP-irf-port1/2]port group interface fortygige 1/1/2
[HP-irf-port1/2]quit
[HP]interface fortygige 1/1/1
[HP-FortyGigE1/1/1]undo shutdown
[HP]interface fortygige 1/1/2
[HP-FortyGigE1/1/2]undo shutdown
[HP]quit
[HP]save
114
High availability networking
[HP]irf-port-configuration active
The following instructions show the steps for creating the IRF stacking ports on switch 2. In this
example, two IRF ports have been used to connect the IRF members to provide redundancy as well
as higher bandwidth (80 GbE). Each IRF port comprises a single 40 GbE port. It is possible to use a
single IRF port to build an IRF instance between the two switches. More than one physical port can be
added to a single logical IRF port so that it can provide higher aggregate bandwidth.
<HP>system-view
[HP]interface fortygige 2/1/1
[HP-FortyGigE2/1/1]shutdown
[HP]interface fortygige 2/1/2
[HP-FortyGigE2/1/2]shutdown
[HP]quit
[HP]irf-port 2/1
[HP-irf-port2/1]port group interface fortygige 2/1/1
[HP-irf-port2/1]quit
[HP]irf-port 2/2
[HP-irf-port2/2]port group interface fortygige 2/1/2
[HP-irf-port2/2]quit
[HP]interface fortygige 2/1/1
[HP-FortyGigE2/1/1]undo shutdown
[HP]interface fortygige 2/1/2
[HP-FortyGigE2/1/2]undo shutdown
[HP]quit
[HP]save
[HP]irf-port-configuration active
16-SFP+ Uplink Module
To create an IRF port using a 16-SFP+ uplink module, at least four 10 GbE ports must be bound to a
logical IRF port. The permitted grouping of these four ports is as follows:
•
Ten-GigabitEthernet1/1/1, Ten-GigabitEthernet1/1/2, Ten-GigabitEthernet1/1/3, TenGigabitEthernet1/1/4
•
Ten-GigabitEthernet1/1/5, Ten-GigabitEthernet1/1/6, Ten-GigabitEthernet1/1/7, TenGigabitEthernet1/1/8
•
Ten-GigabitEthernet1/1/9, Ten-GigabitEthernet1/1/10, Ten-GigabitEthernet1/1/11, TenGigabitEthernet1/1/12
•
Ten-GigabitEthernet1/1/13, Ten-GigabitEthernet1/1/14, Ten-GigabitEthernet1/1/15, TenGigabitEthernet1/1/16
The following example shows how to bind multiple physical interfaces to an IRF port:
[HP]interface ten-gigabitethernet 1/1/1
[Ten-GigabitEthernet1/1/1]shutdown
High availability networking
115
[HP]quit
[HP]interface ten-gigabitethernet 1/1/2
[Ten-GigabitEthernet1/1/2]shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/3
[Ten-GigabitEthernet1/1/3]shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/4
[Ten-GigabitEthernet1/1/4]shutdown
[Ten-GigabitEthernet1/1/4]quit
[HP]irf-port 1/1
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/1
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/2
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/3
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/4
[HP-irf-port1/1]quit
[HP]interface ten-gigabitethernet 1/1/1
[Ten-GigabitEthernet1/1/1]undo shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/2
[Ten-GigabitEthernet1/1/2]undo shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/3
[Ten-GigabitEthernet1/1/3]undo shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/4
[Ten-GigabitEthernet1/1/4]undo shutdown
[Ten-GigabitEthernet1/1/4]quit
[HP]save
[HP]irf-port-configuration active
Repeat the procedure on switch 2 to configure irf-port 2/2. If more than a single IRF port is
needed, configure irf-port 1/2 and irf-port 2/1 as well.
6-SFP Uplink Module
To create an IRF port using a 6-SFP uplink module, at least two 10 GbE ports must be bound to a
logical IRF port. The permitted grouping of these two ports is as follows:
116
High availability networking
•
Ten-GigabitEthernet1/1/1, Ten-GigabitEthernet1/1/2
•
Ten-GigabitEthernet1/1/3, Ten-GigabitEthernet1/1/4
•
Ten-GigabitEthernet1/1/5, Ten-GigabitEthernet1/1/6
The following example shows how to bind multiple physical interfaces to an IRF port:
[HP]interface ten-gigabitethernet 1/1/1
[Ten-GigabitEthernet1/1/1]shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/2
[Ten-GigabitEthernet1/1/2]shutdown
[Ten-GigabitEthernet1/1/1]quit
[HP]irf-port 1/1
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/1
[HP-irf-port1/1]port group interface ten-gigabitethernet 1/1/2
[HP-irf-port1/1]quit
[HP]interface ten-gigabitethernet 1/1/1
[Ten-GigabitEthernet1/1/1]undo shutdown
[HP]quit
[HP]interface ten-gigabitethernet 1/1/2
[Ten-GigabitEthernet1/1/2]undo shutdown
[Ten-GigabitEthernet1/1/1]quit
[HP]save
[HP]irf-port-configuration active
Repeat the procedure on switch 2 to configure irf-port 2/2. If more than a single IRF port is
needed, configure irf-port 1/2 and irf-port 2/1 as well.
5. Connect the cables to the IRF-ports of both switches as defined in the previous example.
The valid connections for 4-QSFP uplink modules are IRF port 1/2 to 2/1, and 1/1 to 2/2. Connecting
IRF port 1/1 to 2/1, or 1/2 to 2/2, will not work. For 6-SFP and 16-SFP+ uplink modules, valid
connections are IRF port 1/1 to 2/2.
The second switch will reboot and take the switch configuration from the master switch (switch 1 in this
example). Also, at this time, if the OS versions are different, the OS on switch 2 will either be upgraded
or downgraded to match the OS version on the master switch.
6. Verify the configuration of the IRF fabric that has been created.
[HP]display irf
MemberID
Role
Priority
CPU-Mac
Description
*+1
Master
1
00e0-fc0f-8c02 --Table Continued
High availability networking
117
2
Standby
1
00e0-fc0f-8c03 ---
----------------------------------------------* indicates the device is the master.
+ indicates the device through which the user logs in.
The Bridge MAC of the IRF is: a0d3-c100-029b
Auto upgrade : yes
Mac persistent : 6 min
Domain ID : 0
[HP]display irf configuration
MemberID
NewID
IRF-Port1
IRF-Port2
1
1
FortyGigE1/1/1
FortyGigE1/1/2
2
2
FortyGigE2/1/1
FortyGigE2/1/2
[HP]display irf topology
Topology Info
----------------------------------------IRF-Port1
IRF-Port2
MemberID
Link
eighbor
Link
neighbor
Belong To
1
DOWN
---
UP
2
00e0fc0f-8c03
2
UP
1
DOWN
---
00e0fc0f-8c03
For more information, see the Moonshot-45XGc Switch IRF Configuration Guide and Moonshot-45XGc
Switch IRF Command Reference on the Hewlett Packard Enterprise website (http://www.hpe.com/info/
moonshot/networking).
Example: Creating an Intelligent Resilient Framework fabric with
Moonshot-45XGc/180XGc switch modules
An IRF fabric can have up to eight stack members. For the example below, the number in the upper left
corner of each uplink module represents the chassis number in the rack and the letter represents the
switch. The IRF numbers shown represent the unit number/IRF port number.
118 Example: Creating an Intelligent Resilient Framework fabric with
Moonshot-45XGc/180XGc switch modules
For more information on stacking Moonshot-45G Switch Modules and Moonshot-180G Switch Modules,
see the Moonshot Switch Module Administrator’s Guide and Moonshot-45G/180G Switch Module CLI
Command Reference documents.
Tips and more information on switch stacking
Following are tips for stacking on Moonshot-45G/180G Switch Modules:
•
Make sure that all the switches in a stack have a unique priority (1 through 15).
•
Make sure that the master switch in a stack has the highest priority of 15.
Following are tips for IRF fabrics on Moonshot-45XGc/180XGc Switch Modules:
•
Make sure that all the switches in an IRF fabric have unique member IDs.
•
Make sure that all the switches in an IRF fabric have a unique priority (1 through 32).
•
Make sure that the master switch in an IRF fabric has the highest priority of 32.
•
Make sure to undo shutdown on the interface ports assigned to IRF ports so that they will link to
neighboring ports in the IRF fabric.
•
Make sure to save when any changes are made, otherwise the changes are lost when the switch is
rebooted.
For more information on IRF fabrics, see the Moonshot-45XGc Switch Series IRF Configuration Guide.
Tips and more information on switch stacking
119
For more information on stacking Moonshot-45G Switch Modules and Moonshot-180G Switch Modules,
see the Moonshot Switch Module Administrator’s Guide and Moonshot-45G/180G Switch Module CLI
Command Reference documents.
For more information on troubleshooting Moonshot-45XGc/180XGc Switch Modules, see the
Moonshot-45XGc Switch Troubleshooting Guide. For more information on general troubleshooting, the
Moonshot System Troubleshooting Guide provides procedures for resolving common problems and
comprehensive courses of action for fault isolation and identification, issue resolution, and software
maintenance on the Moonshot System. These documents are available in the Hewlett Packard Enterprise
Information Library (http://www.hpe.com/info/moonshot/docs).
Switch and server cartridge LAGs
In addition to grouping multiple uplinks on a switch into an aggregated network path to provide additional
bandwidth and/or availability, it is possible to also combine the NICs on a server cartridge to provide
similar functionality. To do this requires special setup of the switches and the server cartridges via the OS
(LAG). The setup of the server side LAGs is presented first. The switch LAG setup follows.
Server cartridge LAG setup
The Linux bonding driver is used to aggregate multiple NICs into a single interface. The driver can be
configured to use one of a number of modes that will distribute traffic according to different characteristics
of each mode. These different modes are listed and described in the following table.
Mode
Name
Description
0
Round-robin (balance-rr)
Transmit network packets in
sequential order from the first
available NIC slave through the
last. This mode provides load
balancing and fault tolerance.
1
Active-backup (active-backup)
Only one NIC slave in the bond is
active. If the active slave fails, a
different slave becomes active.
The single logical bonded
interface MAC address is
externally visible on only
one NIC (port) to avoid distortion
in the network switch. This mode
provides fault tolerance.
2
XOR (balance-xor)
Transmit network packets based
on [(source MAC address XOR'd
with destination MAC
address) modulo NIC slave
count]. This mode selects the
same NIC slave for each
destination MAC address and
provides load balancing and fault
tolerance.
Table Continued
120
Switch and server cartridge LAGs
Mode
Name
Description
3
Broadcast (broadcast)
Transmit network packets on all
slave network interfaces. This
mode provides fault tolerance.
4
IEEE 802.3ad Dynamic link
aggregation (802.3ad)(LACP)
Creates aggregation groups that
share speed and duplex settings.
Utilizes all slave network
interfaces in the active
aggregator group according to
the 802.3ad specification.
5
Adaptive transmit load balancing
(balance-tlb)
Linux bonding driver mode that
does not require any special
network-switch support. The
outgoing network packet traffic is
distributed according to the
current load (computed relative to
the speed) on each network
interface slave. One currently
designated slave network
interface receives incoming
traffic. If this receiving slave fails,
another slave takes over the
MAC address of the failed
receiving slave.
6
Adaptive load balancing
(balance-alb)
Includes balance-tlb plus receive
load balancing (rlb) for IPV4
traffic, and does not require any
special network switch support.
ARP negotiation achieves receive
load balancing. The bonding
driver intercepts the ARP Replies
sent by the local system on their
way out and overwrites the
source hardware address with
the unique hardware address of
one of the NIC slaves in the
single logical bonded interface
such that different network-peers
use different MAC addresses for
their network packet traffic.
Reference:
Mode 4, identified in the previous table as LACP/802.3ad, is the mode used for Dynamic Link
Aggregation. The other modes are used to configure a Static Link Aggregation. Modes 0 (balance-rr), 2
(balance-xor), 3 (broadcast), and 4 (LACP/802.3ad) require that the two chassis switches be stacked into
a single logical switch. The other modes may be configured with switches set to operate independently.
Example: Configuring link aggregation on RHEL (+CentOS, Fedora, and so on)
Use the following instructions to configure link aggregation on RHEL.
#Configuring Link Aggregation on RHEL (and CentOS, Fedora, and so on)
Example: Configuring link aggregation on RHEL (+CentOS, Fedora, and so
on) 121
All of the RHEL and related Linux distributions configure Link aggregation in a similar fashion, so these
steps would apply to them all:
#First, create a file named /etc/sysconfig/network-scripts/ifcfg-bond0 with the following contents:
DEVICE=bond0
ONBOOT=yes
USERCTL=no
#You can also configure your bond interface to specify DHCP or static IP addressing as you would other
standard Ethernet interfaces. For example, to configure your bonded interface to have a static IP address,
you would modify this file to contain:
DEVICE=bond0
ONBOOT=yes
USERCTL=no
BOOTPROTO=static
IPADDR=192.168.1.10
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
#Or to configure it for DHCP:
DEVICE=bond0
ONBOOT=yes
USERCTL=no
BOOTPROTO=dhcp
DHCP_HOSTNAME=myhostname
#Next, edit the interface configuration for each NIC you wish to include in the bonded interface. With
Moonshot cartridges, which will mean /etc/sysconfig/ifcfg-eth0 and /etc/sysconfig/ifcfg-eth1 usually. Edit
these to contain
DEVICE=ethX
ONBOOT=yes
BOOTPROTO=manual
MASTER=bond0
SLAVE=yes
USERCTL=no
Replacing the ‘X’ with the appropriate interface number.
#Configure the bonding mode options by editing or created /etc/modprobe.conf to include the following:
alias bond0 bonding
options bond0 miimon=100 mode=0
#To configure your bonded interface to use the first mode, bonding-rr (round-robin.) You can alter this
mode to use one of the other modes described previously. For instance, to use LACP (and to specify an
additional parameter) you would adjust the same file to include:
alias bond0 bonding
options bond0 miimon=100 mode=4 lacp_rate=1
Test your configuration by loading the bonding module with the modprobe bond0 command and restart
your networking with the service network restart command.
Example: Configuring link aggregation on Ubuntu
Use the following instructions to configure link aggregation on Ubuntu.
122
Example: Configuring link aggregation on Ubuntu
To configure Link Aggregation on Ubuntu, simply edit the /etc/network/interfaces file to include
the following:
auto eth0
iface eth0 inet manual
bond-master bond0
auto eth1
iface eth1 inet manual
bond-master bond0
auto bond0
iface bond0 inet static
address 192.168.1.10
netmask 255.255.255.0
gateway 192.168.1.1
bond-mode bonding-rr
bond-miimon 100
(You can specify the mode as a name, or a number.) Or to enable dynamic LACP bonding:
auto eth0
iface eth0 inet manual
bond-master bond0
auto eth1
iface eth1 inet manual
bond-master bond0
auto bond0
iface bond0 inet static
address 192.168.1.10
netmask 255.255.255.0
gateway 192.168.1.1
bond-mode 4
bond-miimon 100
bond-lacp-rate 1
Test your new bonded interface by restarting your networking services with the command service
network restart.
Example: Configuring link aggregation on SLES
Configuring link aggregation on SLES is done in a similar fashion, by editing a few network configuration
files, and executing a couple of commands (or just rebooting.)
First, create a file named /etc/sysconfig/network/ifcfg-bond0 with the following contents:
BOOTPROTO='static'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=’192.168.1.10/24’
MTU=''
NAME=''
NETWORK=''
Example: Configuring link aggregation on SLES
123
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=active-backup miimon=100'
BONDING_SLAVE_0='eth0'
BONDING_SLAVE_1='eth1'
You can configure this new interface to use a static IP, or DHCP as you require. You can also choose a
different bonding mode using the table above to determine the name or mode number you want.
Then edit both of the NIC configuration files (/etc/sysconfig/network/ifcfg-eth0 and ifcfg-eth1) to contain:
BOOTPROTO='none'
IPADDR=''
BROADCAST=''
STARTMODE='hotplug'
USERCONTROL='no'
ETHTOOL_OPTIONS=''
MTU=''
NETWORK=''
REMOTE_IPADDR=''
Disable your current network by shutting down both Ethernet interfaces with the commands ifdown
eth0, and ifdown eth1 and then start the new bonded interface with the command ifup bond0.
Switch LAG setup
This section explains how to create a static LAG. A LAG is the combining of multiple physical network
interfaces in parallel in order to increase the throughput beyond that of what a single network interface
can sustain. There are two main types of LAGs supported by the Moonshot switches: static and dynamic.
A dynamic LAG uses a LACP packet exchange between the two LAG partners. A static LAG does not
use the LACP packet exchange between the two LAG partners. It relies on the user to configure both
LAG partners identically to ensure proper operation of the LAG.
The following drawing shows a simple example of a LAG of two uplink ports of a Moonshot switch. Only
two ports (or interfaces) are shown for simplicity.
124
Switch LAG setup
Example: Creating static LAG with Moonshot-45G/180G switches
The following steps are used to create a static LAG of two physical uplink ports as shown in the previous
illustration with these types of switch modules.
Procedure
1. Enter the following commands.
(Routing) >en
(Routing) #config
(Routing) (Config)#interface 1/1/1,1/1/2
(Routing) (Interface 1/1/1,1/1/2)#addport 0/3/1
(Routing) (Interface 1/1/1,1/1/2)#no port lacpmode
(Routing) (Interface 1/1/1,1/1/2)#interface 0/3/1
(Routing) (Interface 0/3/1)#port-channel static
(Routing) (Interface 0/3/1)#exit
(Routing) (Config)#exit
(Routing) #
2. Connect the two uplink ports of the newly formed LAG to the two ports of the remote switch. Make
sure that the two ports of the connecting switch have the same speed configured as a static LAG.
3. Save the configuration.
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n) y
Config file 'startup-config' created successfully.
(Routing) #
Example: Creating static LAG with Moonshot-45G/180G switches
125
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Creating static LAG with Moonshot-45XGc/180XGc switches
The following steps are used to create a static LAG of two physical uplink ports shown in the preceding
example with these types of switch modules.
Procedure
1. Enter the following commands.
<HP>system-view
[HP]interface bridge-aggregation 1
[HP-Bridge-Aggregation1]quit
[HP]interface fortygige 1/1/1
[HP-FortyGigE1/1/1]port link-aggregation group 1
[HP-FortyGigE1/1/1]interface fortygige 1/1/2
[HP-FortyGigE1/1/2]port link-aggregation group 1
[HP-FortyGigE1/1/2]quit
[HP]interface bridge-aggregation 1
[HP-Bridge-Aggregation1]port link-type trunk
[HP-Bridge-Aggregation1]port trunk permit vlan all
NOTE:
The vlan can be an integer 1 to 4096 or all.
[HP-Bridge-Aggregation1]quit
[HP]
2. Connect the two uplink ports of the newly formed LAG to the two ports of the remote switch.
Make sure that the two ports of the connecting switch are of the same speed configured as a static
LAG.
3. Save the configuration.
<HP>save
The current configuration will be written to the device. Are you sure?
[Y/N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
126
Example: Creating static LAG with Moonshot-45XGc/180XGc switches
Example: Creating dynamic LAG with Moonshot-45G/180G switches
The following steps are used to create a dynamic LAG of two physical uplink ports shown in the
preceding example with these types of switch modules.
Procedure
1. Enter the following commands.
(Routing) >en
(Routing) #config
(Routing) (Config)#interface 1/1/1,1/1/2
(Routing) (Interface 1/1/1,1/1/2)#addport 0/3/1
(Routing) (Interface 1/1/1,1/1/2)#port lacpmode
(Routing) (Interface 1/1/1,1/1/2)#interface 0/3/1
(Routing) (Interface 0/3/1)#no port-channel static
(Routing) (Interface 0/3/1)#exit
(Routing) (Config)#exit
(Routing) #
2. Connect the two uplink ports of the newly formed LAG to the two ports of the remote switch.
NOTE:
Be sure that the connecting switch has two ports of similar type configured as a dynamic LAG.
3. Save the configuration.
(Routing) #write memory
This operation may take a few minutes.
Management interfaces will not be available during this time.
Are you sure you want to save? (y/n)y
Config file 'startup-config' created successfully.
(Routing) #
For more information, see the Moonshot-45G/180G Switch Module CLI Command Reference Guide on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Example: Creating dynamic LAG with Moonshot-45XGc/180XGc switches
The following steps are used to create a dynamic LAG of two physical uplink ports shown in the
preceding example with these types of switch modules.
Procedure
1. Enter the following commands.
<HP>system-view
[HP]interface bridge-aggregation 1
Example: Creating dynamic LAG with Moonshot-45G/180G switches
127
[HP-Bridge-Aggregation1]link-aggregation mode dynamic
[HP-Bridge-Aggregation1]interface fortygige 1/1/1
[HP-FortyGigE1/1/1]port link-aggregation group 1
[HP-FortyGigE1/1/1]interface fortygige 1/1/2
[HP-FortyGigE1/1/2]port link-aggregation group 1
[HP-FortyGigE1/1/2]quit
[HP]interface bridge-aggregation 1
[HP-Bridge-Aggregation1]port link-type trunk
[HP-Bridge-Aggregation1]port trunk permit vlan all
NOTE:
The vlan can be an integer 1 to 4096 or all.
[HP-Bridge-Aggregation1]quit
[HP]
2. Connect the two uplink ports of the newly formed LAG to the two ports of the remote switch.
NOTE:
Be sure that the connecting switch has two ports of similar type configured as a dynamic LAG.
3. Save the configuration.
<HP>save
The current configuration will be written to the device. Are you sure?
[Y/N]:y
Please input the file name(*.cfg)[flash:/startup.cfg]
(To leave the existing filename unchanged, press the enter key):
flash:/startup.cfg exists, overwrite? [Y/N]:y
Validating file. Please wait...
Saved the current configuration to mainboard device successfully.
<HP>
For more information, see the Moonshot-45XGc Switch Layer 2 – LAN Switching Command Reference on
the Hewlett Packard Enterprise website (http://www.hpe.com/info/moonshot/networking).
Tips and more information on switch LAG setup
Following are general tips for LAGs on both Moonshot-45G/180G and Moonshot-45XGc/180XGc Switch
Modules:
128
•
Each member of the LAG must be running the same speed and must be in full duplex mode.
•
The configured speed of a LAG member cannot be changed.
•
The port cannot be a mirrored port.
Tips and more information on switch LAG setup
•
A port can only be a member of one LAG.
•
Make sure that the LAG ports on the connecting switch are configured the same as the newly created
LAG ports. Do not intermix static LAGs and dynamic LAGs.
Tip for LAGs on Moonshot-45G/180G Switch Modules:
Make sure that the LAG port used is in the form of x/3/x.
Tips for LAGs on Moonshot-45XGc/180XGc Switch Modules:
•
Make sure to choose a bridge-aggregation interface number that is not already used on the switch.
The display link-aggregation summary command shows all bridge-aggregation (BAGG) interfaces that
are being used, if any.
•
Make sure to add any VLANs (other than the default of VLAN 1) required on this LAG.
For more information on troubleshooting Moonshot-45XGc/180XGc Switch Modules, see the
Moonshot-45XGc Switch Troubleshooting Guide. For more information on general troubleshooting, the
Moonshot System Troubleshooting Guide provides procedures for resolving common problems and
comprehensive courses of action for fault isolation and identification, issue resolution, and software
maintenance on the Moonshot System. These document are available in the Hewlett Packard
Enterprise Information Library.
PXE support when LACP is configured
When LACP is configured between a cartridge and a switch but the cartridge is booting up with PXE,
LACP will not run on the cartridge. Complete the following to ensure that packet communications will
occur even with a lack of a successful LACP connection between the cartridge and the switch:
•
45G and 180G switches: To allow communication through the switch when the cartridge is PXEbooting, ensure that the firmware version on the switches is 2.0.3 or later.
•
45XGc, 180XGc, and 45Gc switches: To allow communication through the switch when the cartridge is
PXE-booting, use the following configuration on the switch on each aggregation between the switch
and the cartridge (In this example, it is configured on a single LACP instance, "bridge-aggregation 1"):
[HP]interface bridge-aggregation 1
[HP-Bridge-Aggregation1]lacp edge-port
Troubleshooting server cartridge LAG setup
Check for bonding errors by running the command "cat /proc/net/bonding/bond0". This action
displays the current status of the aggregated connection and additional details, depending on the mode
chosen.
PXE support when LACP is configured
129
Support and other resources
Accessing Hewlett Packard Enterprise Support
•
For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website:
http://www.hpe.com/assistance
•
To access documentation and support services, go to the Hewlett Packard Enterprise Support Center
website:
http://www.hpe.com/support/hpesc
Information to collect
•
Technical support registration number (if applicable)
•
Product name, model or version, and serial number
•
Operating system name and version
•
Firmware version
•
Error messages
•
Product-specific reports and logs
•
Add-on products or components
•
Third-party products or components
Accessing updates
•
Some software products provide a mechanism for accessing software updates through the product
interface. Review your product documentation to identify the recommended software update method.
•
To download product updates:
Hewlett Packard Enterprise Support Center
www.hpe.com/support/hpesc
Hewlett Packard Enterprise Support Center: Software downloads
www.hpe.com/support/downloads
Software Depot
www.hpe.com/support/softwaredepot
•
To subscribe to eNewsletters and alerts:
www.hpe.com/support/e-updates
•
130
To view and update your entitlements, and to link your contracts and warranties with your profile, go to
the Hewlett Packard Enterprise Support Center More Information on Access to Support Materials
page:
Support and other resources
www.hpe.com/support/AccessToSupportMaterials
IMPORTANT:
Access to some updates might require product entitlement when accessed through the Hewlett
Packard Enterprise Support Center. You must have an HPE Passport set up with relevant
entitlements.
Customer self repair
Hewlett Packard Enterprise customer self repair (CSR) programs allow you to repair your product. If a
CSR part needs to be replaced, it will be shipped directly to you so that you can install it at your
convenience. Some parts do not qualify for CSR. Your Hewlett Packard Enterprise authorized service
provider will determine whether a repair can be accomplished by CSR.
For more information about CSR, contact your local service provider or go to the CSR website:
http://www.hpe.com/support/selfrepair
Remote support
Remote support is available with supported devices as part of your warranty or contractual support
agreement. It provides intelligent event diagnosis, and automatic, secure submission of hardware event
notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolution based on your
product's service level. Hewlett Packard Enterprise strongly recommends that you register your device for
remote support.
If your product includes additional remote support details, use search to locate that information.
Remote support and Proactive Care information
HPE Get Connected
www.hpe.com/services/getconnected
HPE Proactive Care services
www.hpe.com/services/proactivecare
HPE Proactive Care service: Supported products list
www.hpe.com/services/proactivecaresupportedproducts
HPE Proactive Care advanced service: Supported products list
www.hpe.com/services/proactivecareadvancedsupportedproducts
Proactive Care customer information
Proactive Care central
www.hpe.com/services/proactivecarecentral
Proactive Care service activation
www.hpe.com/services/proactivecarecentralgetstarted
Warranty information
To view the warranty for your product or to view the Safety and Compliance Information for Server,
Storage, Power, Networking, and Rack Products reference document, go to the Enterprise Safety and
Compliance website:
www.hpe.com/support/Safety-Compliance-EnterpriseProducts
Customer self repair
131
Additional warranty information
HPE ProLiant and x86 Servers and Options
www.hpe.com/support/ProLiantServers-Warranties
HPE Enterprise Servers
www.hpe.com/support/EnterpriseServers-Warranties
HPE Storage Products
www.hpe.com/support/Storage-Warranties
HPE Networking Products
www.hpe.com/support/Networking-Warranties
Regulatory information
To view the regulatory information for your product, view the Safety and Compliance Information for
Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise
Support Center:
www.hpe.com/support/Safety-Compliance-EnterpriseProducts
Additional regulatory information
Hewlett Packard Enterprise is committed to providing our customers with information about the chemical
substances in our products as needed to comply with legal requirements such as REACH (Regulation EC
No 1907/2006 of the European Parliament and the Council). A chemical information report for this product
can be found at:
www.hpe.com/info/reach
For Hewlett Packard Enterprise product environmental and safety information and compliance data,
including RoHS and REACH, see:
www.hpe.com/info/ecodata
For Hewlett Packard Enterprise environmental information, including company programs, product
recycling, and energy efficiency, see:
www.hpe.com/info/environment
Documentation feedback
Hewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us
improve the documentation, send any errors, suggestions, or comments to Documentation Feedback
(docsfeedback@hpe.com). When submitting your feedback, include the document title, part number,
edition, and publication date located on the front cover of the document. For online help content, include
the product name, product version, help edition, and publication date located on the legal notices page.
132
Regulatory information
Acronyms and abbreviations
AOC
active optical cable
BPDU
Bridge Protocol Data Unit
CM
chassis manager
DAC
direct attach cable
ESD
electrostatic discharge
FC
Fibre Channel
GbE
gigabit Ethernet
GBIC
Gigabit Interface Converter
HPE APM
HPE Advanced Power Manager
iLO
Integrated Lights-Out
IRF
Intelligent Resilient Framework
LACP
Link Aggregation Control Protocol
LAG
link aggregation group
LC
Lucent connector
MPO
multifiber push-on
NOS
network operating system
PXE
preboot execution environment
QoS
Acronyms and abbreviations
133
Quality of Service
QSFP+
enhanced quad small form-factor pluggable
SCP
Secure Copy Protocol
SD
Secure Digital
SFP+
enhanced small form-factor pluggable
SLES
SUSE Linux Enterprise Server
SoC
system on chip
SR
short range
SSH
Secure Shell
SX
short haul
TFTP
Trivial File Transfer Protocol
ToR
top of rack
TRILL
Transparent Interconnection of Lots of Links
USB
universal serial bus
VLAN
virtual local-area network
VNC
virtual network computing
VSP
virtual serial port
VTY
Virtual Type terminal user interface
134
Acronyms and abbreviations