2–Configuring NIC
Teaming/Bonding
Teaming/Bonding
Windows Teaming
You can group together multiple network adapters in a server to make a team.
Individual adapters that are part of a team operate as a team rather than standalone adapters. A team provides traffic load balancing across the member adapters and fault tolerance when some, but not all, of the members lose connectivity.
To enable teaming functionality, install the teaming driver in addition to the basic
NIC.
Team MAC Address
At initialization, the teaming driver selects the team’s MAC address to be the MAC of one of the teamed adapters. In general, the first adapter to come up is chosen as the preferred primary adapter. The preferred primary’s MAC address is assigned to the MAC address of the team. Alternately, you can choose any valid
MAC address as the team’s static MAC address, also called the locally administered address (LAA). Make sure any provided LAA is unique for the local
Ethernet network. This provision gives the system administrator more flexibility in configuring the MAC address for a team when necessary.
Teaming Modes
Teaming is designed to improve reliability and fault tolerance of networks and to enhance performance by efficient load balancing.
The following NIC teaming modes are provided:
Failsafe Mode ensures that an alternate standby or redundant adapter becomes active if the primary network connection fails.
Switch Independent Load-Balancing Mode ensures distribution of transmit loads across the teamed adapters.
Link Aggregation Modes (802.3ad static, 802.3ad dynamic (active and passive link aggregation control protocol [LACP])) enables the use of multiple adapters together as a single, virtual adapter with the aggregated capacity of its individual adapters.
All team types—failsafe, switch-independent load balancing, and link aggregation—can be heterogeneous as well as homogeneous. Every team must have at least one QLogic Adapter.
2-14 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Mode
Failsafe
Table 2-2 shows that failsafe and transmit load-balancing modes are switch
independent, which means they do not require switch configuration. LACP or
802.3ad require switch ports configured for LACP.
Table 2-2. Windows Teaming Modes
Failover
Capability
Yes: Layer 2
Switch
Dependency
No
System Fault
Tolerance
(SFT)
Yes
Load
Balancing
No
Number of
Ports per Team
(Range
a
)
2–16
Transmit load balancing
Static
802.3ad
Yes No Yes
Yes Yes Yes
3 or 4
Yes
Dynamic
802.3ad
Yes Yes Yes Yes a
16×16 ports can be aggregated per system: 16 ports per team and 16 teams per system.
2–16
2–16
Failsafe Mode
The failsafe mode provides Layer 2 fault tolerance. Failsafe provides high reliability through redundancy in the event of port failure. When the primary network connection is down, data traffic is automatically transferred to a secondary, standby connection. The preferred primary adapter can be specified either by the system administrator or by the teaming driver (if the admin does not select the preferred adapter). When the teaming driver needs to make the selection, it selects the best adapter in terms of bandwidth, health, and capability.
The preferred primary must always be a QLogic Adapter.
The administrator can also choose one of the following failback types to specify the behavior when connection to the preferred primary is restored after a period of failure:
None—When the preferred primary becomes operational again, the driver does not automatically switch back the primary to the active adapter.
Preferred Primary—When the preferred primary becomes operational again, the driver automatically switches back the primary as the active adapter. The network traffic resumes to the primary adapter from the standby adapter. The traffic stays with the secondary adapter only as long as the primary adapter is down.
SN0054671-00 D 2-15
2–Configuring NIC
Teaming/Bonding
Auto Select—Use this option to enable the teaming driver to automatically select the best adapter based on parameters such as bandwidth, link state, health, and so on.
In failsafe mode, the standby adapter could be dissimilar in the individual features supported and capacity, and may come from a different vendor.
All the adapters in the team share a common team MAC address. This is either a locally administered MAC address or a default MAC address specified by the driver. Only one adapter at a time in the team is active for network traffic. No two same MAC addresses are exposed to the switch at the same time.
Failsafe mode is inherent in all other teaming modes and is switch agnostic.
Switch-Independent Load Balancing Mode
Switch-independent load balancing mode provides a failsafe feature and supports transmit load balancing. For receive load balancing, use the 802.3ad modes.
In this mode, the outbound traffic is efficiently distributed across the member adapters to increase the transmit bandwidth. Traffic load balancing is connection-based to avoid out-of-order packet delivery. The administrator can select one of the following load distribution types:
Auto Select indicates that the load is distributed based on the target IP address (IPv4 or IPv6) and port number. This option ensures a one-to-one correspondence between a traffic flow and a team adapter.
MAC address based indicates that the load is distributed based on the target MAC address.
In switch-independent load balancing, a team receives the traffic on the preferred primary adapter. If the preferred primary adapter fails, the receive load switches to a secondary adapter (failover operation). If the preferred primary adapter becomes operational again, the receive load fails back to the preferred primary adapter (failback operation). Thus, a switch-independent load balancing team also behaves like a failsafe team. Each time the preferred primary changes due to failover or failback, other network elements are notified of the change in the primary adapter through team gratuitous address resolution protocols (ARPs).
Link Aggregation Mode
link aggregation provides increased bandwidth and high reliability by combining several NICs into a single, logical, network interface called a link aggregation group (LAG). The link aggregation is scalable, meaning an adapter can be added or deleted either statically or dynamically from a team.
Traffic from all the team ports that form a LAG have the same MAC address, which is the MAC address of the team. If a new adapter joins the LAG, or an adapter forming the LAG fails, the LAG becomes operational again after a brief exchange of protocols between the switch and the server. QLogic Adapters are rapidly aggregated, with a latency of 1 to 2 seconds.
2-16 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Two options are available in the link aggregation mode:
Static link aggregation
Dynamic link aggregation
NOTE
The switch must support the IEEE 802.3ad standard for the preceding two link aggregation modes to work.
Static Link Aggregation (SLA)
Static link aggregation (SLA, 802.3ad static protocols with generic trunking) is a switch-assisted teaming mode, where the switch must be 802.3ad compliant. The switch ports must be configured so that the switch perceives adapters from a LAG as a single, virtual adapter.
In SLA, the ports on the switch are active by default. There is no negotiation between the switch and the teaming driver to decide on adapters participating in a LAG.
In SLA mode, the protocol stack responds to ARP requests with a single, advertised MAC address, and an IP address corresponding to the LAG. Each physical adapter in the team uses the same team MAC address during transmission. As the switch (at the other end of link) is aware of the trunking teaming mode, it appropriately modifies the forwarding table to indicate the trunk as a single virtual port. This modification ensures correct traffic routing on the receive side as well. In this mode, the switch also distributes receive traffic across the member adapters.
Dynamic Link Aggregation (DLA)
Dynamic link aggregation (DLA) with LACP is similar to SLA except that LACP allows self configuration of LAG through handshaking between the switch and the intermediate driver. For the team to function, LACP must be enabled at both ends of the link: the server and the switch. LACP (802.3ad dynamic) allows switch ports to dynamically communicate with the teaming driver, allowing controlled addition and removal of ports from the team.
Link aggregation mode has transmit load balancing and fail safety support. If a link connected through a participant port of a link-aggregated team goes down,
LACP provides failover and load balancing across the remaining members of the team. In addition, if a new member port is added to the team or is removed from the team, the switch performs load rebalancing for the receive operation and the driver performs load balancing for the transmit operation, to accommodate the change in configuration.
SN0054671-00 D 2-17
2–Configuring NIC
Teaming/Bonding
Transmit load distribution in LACP provides the following options:
None indicates no traffic distribution. Only a single “active” adapter is used for transmit. The driver selects the active adapter based on LACP state information.
Auto Select indicates that the load is distributed based on the target IP address and port number. This option ensures a one-to-one correspondence between a traffic flow and a team adapter.
MAC address based indicates that the load is distributed based on the target MAC address.
Using the CLI for Teaming
You can view, create, configure, and delete teams using the QConvergeConsole
CLI utility.
To view a list of teams, issue the following command:
qaucli -nic –teamlist
To view team information, issue the following command:
qaucli -nic -teaminfo <team_inst|ALL>
To preview available ports before configuring a new team, issue the following command:
qaucli -nic -teamnew_portspreview
To configure a new team, issue the following command:
qaucli -nic -teamnew <team_type> <port_insts|ALL>
where port_insts
are the ports indices separated by commas (for example, 1,2) and
team_type
is either
1=Fail Over
or
2=Load Balanced
.
To delete a team, issue the following command:
qaucli -nic -teamdel <team_inst|ALL>
2-18 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Using the Team Management GUI
Use the Team Management property page to manage the following teaming-related activities:
Viewing network topology
Creating, modifying, and deleting teams
Viewing and changing team properties
Adding and deleting virtual adapters
To open the Team Management property page:
1.
In Windows, access the Computer Management dialog box, and then click
Device Manager in the left pane.
2.
Under Network adapters, right-click the QLogic 10Gigabit Ethernet adapter, and then select Properties.
3.
Click the Team Management tab to bring that page to the front (
Figure 2-5
) and perform teaming-related management.
SN0054671-00 D
Figure 2-5. Team Management Property Page
2-19
2–Configuring NIC
Teaming/Bonding
On the Team Management page, the Teams and Adapters pane on the left lists the network devices currently present on this system, including:
Teams and virtual adapters, as well as their member physical adapters
QLogic and other vendor adapters
Teaming Configuration
Teaming configuration includes creating, modifying, and deleting teams, and viewing team statistics on the Team Management property page. To launch the
Team Management property page, see “Using the Team Management GUI” on page 2-19 .
Information on teaming configuration includes the following:
“Creating a Team” on page 2-21
“Modifying a Team” on page 2-27
“Deleting a Team” on page 2-32
“Saving and Restoring Teaming Configuration” on page 2-33
2-20 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Creating a Team
To create a team:
1.
To create a team, right-click the Teams folder icon, and then click Create
Team (
Figure 2-6 ).
Figure 2-6. Creating a Team
2.
The software automatically picks a unique team name, or you can enter your own team name. Team names must be unique on a system.
SN0054671-00 D 2-21
2–Configuring NIC
Teaming/Bonding
3.
On the Create Team dialog box, specify the following (see the message pane at the bottom of the dialog box for more details), and then click OK to return to the adapter properties:
Name—Type a name for the new team.
Type—Select the teaming mode by clicking either Failsafe Team,
802.3ad Static Team, 802.3ad Dynamic Team, or Switch
Independent Load Balancing. If you select the 802.3ad dynamic option, you must also select one of the following options:
Active LACP: LACP is a Layer 2 protocol that controls the teaming of physical ports into an aggregated set. LACP discovers if a host’s ports are connected to a switch that supports aggregation on the connected ports and configures those ports into an aggregation bundle. For LACP to operate, one side has to be Active LACP. The Active LACP side of the protocol initiates the protocol.
Passive LACP: The Passive LACP side responds to the active
LACP requests.
Adapters to Add—Select the check box next to each adapter that should form the team.
Use default MAC Address—Select this check box to have the driver assign a MAC address, or clear the check box to select a locally-administered MAC address from the list.
Select Preferred Primary Adapter—Choose a preferred primary adapter for the team from the list of teamed adapters, or None to allow the driver to assign the preferred primary adapter.
Failback Type—If this is a Failsafe Team, select a failback type of
None, Auto Select, or Preferred Primary.
Load Balancing Type—If this is an 802.3ad Static Team or 802.3ad
Dynamic Team, select the type of load balancing: Auto, MAC Address
Based, or None.
Distribution Type—If this is a Switch Independent Load Balancing team type, select a distribution type of either Auto Select or MAC
Address Based.
Advanced—Click this button to configure QLogic-specific team capabilities such as RSS, MTU, or various offloads. These properties configure the member adapters to avoid any conflict after a team has been created.
2-22 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Figures 2-7
through 2-10
show the configuration of various teaming modes.
Figure 2-7. Creating a Failsafe Team
SN0054671-00 D 2-23
2–Configuring NIC
Teaming/Bonding
Figure 2-8. Creating a Switch-Independent Load Balancing Team
2-24 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Figure 2-9. Creating an 802.3ad Static Team
SN0054671-00 D 2-25
2–Configuring NIC
Teaming/Bonding
Figure 2-10. Creating an 802.3ad Dynamic Team
2-26 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
To confirm if a team has been successfully created, view the Team and Adapters pane on the Team Management page.
Figure 2-11
shows an example of a newly-formed team. The Team Data pane on the right shows the properties, information, and status of the team or adapter that is currently selected in the Teams and Adapters pane on the left.
Figure 2-11. Confirming New Team Creation
Modifying a Team
A team can be modified by:
Adding or removing one or more team members to a team.
Modifying the team properties.
SN0054671-00 D 2-27
2–Configuring NIC
Teaming/Bonding
To add team members:
1.
On the Team Management property page, right-click the unteamed adapter to add to a team.
2.
On the shortcut menu, point to Add to Team, and then click the team to which you want to add the adapter (
Figure 2-12 ).
NOTE
You cannot add an adapter to a team that is already a member of another team. Teaming of teams is not supported.
2-28
Figure 2-12. Adding a Team
SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
To remove an adapter from a team:
NOTE
A team must include at least one QLogic Adapter. A QLogic Adapter is allowed to be deleted from a team only if it is not the last QLogic-teamed
Adapter.
1.
On the Team Management property page, right-click the adapter to be removed from the team.
2.
On the shortcut menu, click Remove from Team.
3.
At least two adapters must be present in a team. If an adapter is no longer required to be a member of a team, it can be removed from the team.
To change a team property:
For the VLAN and teaming solution to work correctly, the properties of all teamed adapters and adapters with multiple VLANs must remain synchronized with the team properties. Ensure that you change the properties of a team and an adapter with VLANs only on the Team Management page.
1.
On the Team Management page, in the right pane under Team Data, expand the Properties list.
2.
Double-click the team property you need to change.
3.
In the Advanced Team Properties dialog box (
Figure 2-13 ), specify a new
property value, and then click OK.
SN0054671-00 D
Figure 2-13. Modifying Advanced Team Properties
2-29
2–Configuring NIC
Teaming/Bonding
The team properties change takes effect immediately. Changing team properties causes the driver to reload, which could result in a momentary loss of connectivity.
NOTE
To ensure that the properties of all teamed adapters and adapters with
VLANs remain synchronized with the team properties, do not directly modify the adapter properties on the Advanced page. If an adapter property becomes out of sync with its team properties, change either the team or adapter property so that they are the same on each, and then reload the team. To reload a team: On the Team Management page, in the left pane under Teams and Adapters, right-click the team name, and then click
Reload Team.
To modify team composition:
1.
On the Team Management page, in the left pane under Teams and
Adapters, right-click the team name whose properties are to be changed.
2.
On the shortcut menu, click Modify Team (
Figure 2-14 ).
2-30
Figure 2-14. Modifying Team Properties
SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
3.
In the Modify Team Properties dialog box, change the team parameters as needed, and then click OK.
Example 1
For a failsafe team, you can change the team name, assigned team static MAC address, preferred primary adapter, and failback type (
Figure 2-15 ).
Figure 2-15. Modifying Failsafe Team Properties
SN0054671-00 D 2-31
2–Configuring NIC
Teaming/Bonding
Example 2
You can change the team type and the corresponding team attributes. For example, you can change from failsafe to switch-independent load balancing, or from 802.3ad static team to 802.3ad dynamic team.
Figure 2-16
shows a failsafe team modification, which shows the new team type and default values for team attributes. You can manually change attribute values.
Figure 2-16. Modifying the Team Type
Deleting a Team
To delete a team:
1.
On the Team Management property page, in the left pane under Teams
and Adapters, right-click the team name to be deleted.
2.
On the shortcut menu, click Delete team.
2-32 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Saving and Restoring Teaming Configuration
QLogic recommends that you periodically save the configuration to prevent any accidental loss of network topology and settings. Current configuration including the teams, VLANs, and properties can be saved to a file. Restoring an earlier configuration results in the destruction of the current configuration.
To save a configuration:
1.
On the Team Management page under Teams and Adapters, right-click the Teams folder.
2.
On the shortcut menu, click Save to File.
3.
Enter a location to save the configuration.
To restore a configuration:
1.
On the Team Management page under Teams and Adapters, right-click the Teams folder.
2.
On the shortcut menu, click Restore From File.
3.
Select a previously saved configuration file.
Viewing Teaming Statistics
You can view teaming and Ethernet statistics using the QConvergeConsole
(QCC) CLI utility.
To view teaming statistics, issue the following command:
qaucli -nic -statport
To reset the Ethernet statistics counter, issue the following command:
qaucli -nic -sreset [cna_port_inst]
To display Ethernet port statistics, issue the following command:
qaucli -nic -statport [cna_port_inst]
To undo the reset of Ethernet statistics counters, issue the following command:
qaucli -nic -sunreset [cna_port_inst]
Linux Bonding/Failover/Aggregation
The Linux qlcnic
driver supports all the standard bonding modes supported by the Linux bonding driver for bonding, failover, and aggregation. For additional details on the bonding modes, refer to the Linux bonding information in the Red
Hat and Novell documentation for your Linux distro version.
SN0054671-00 D 2-33
2–Configuring NIC
Teaming/Bonding
Using LACP on 8200 Series Adapters for Windows
NOTE
This feature is available only on Cisco
®
systems (Cisco FCoE switch).
By default, link aggregation control protocol (LACP) is disabled on 8200 Series
Adapters (it is enabled by default in the 3200 Series Adapters).
Perform the following steps using QLogic Device Windows Properties Page to enable LACP on a 8200 Series Adapter.
1.
Log in to the server that contains installed 8200 Series Adapters.
2.
Open the Server Manager and select Diagnostics > Device Manager >
Network Adapters.
3.
Right-click the first QLogic 10 Gigabit Ethernet CNA device and select
Properties from the context menu (
Figure 2-39 ).
4.
Click the Team Management tab.
5.
In Teams and Adapters, click Teams, and then click Custom Settings.
The Teaming custom settings dialog box displays (
Figure 2-17 ).
2-34
Figure 2-17. Enabling LACP
6.
Select the Allow 802.3as team over CNA check box, and then click OK.
The network must be configured to a Cisco-supported configuration, as described in the following paragraphs. This information is from the Cisco support forum thread, https://supportforums.cisco.com/thread/2071713 .
SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
A virtual fabric channel (vFC) can be bound inside a virtual port channel (vPC) because the Cicso Nexus ® 5x00 (FCF) sees only one link.
Figure 2-18
illustrates a valid configuration with one vPC.
Figure 2-18. LACP Configuration with One vPC
The vFC must be bound by a physical interface in a vPC configuration.
SN0054671-00 D 2-35
2–Configuring NIC
Teaming/Bonding
If a server has four Converged Network Adapter ports, then two vPCs can be created, as shown in
Figure 2-19 .
Figure 2-19. LACP Configuration with Two vPCs
An vFC cannot be bound from a server with multiple links in the same
Nexus 5x00.
The configurations in
Figure 2-20
and Figure 2-21 are unsupported because the
vPC can be bound to only one interface.
Figure 2-20. Invalid LACP Configuration (One vPC Linked to Two Nexus 5x00s)
2-36 SN0054671-00 D
2–Configuring NIC
Teaming/Bonding
Figure 2-21. Invalid LACP Configuration (Two vPCs Linked Separately to Two
Nexus 5x00s)
For information on configuring Cisco Nexus switches for vPCs, see to the following:
Cisco Nexus 5000 Series Configuring vPCs at:
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/ interfaces/configuration/guide/if_vPC.pdf
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/ configuration_guide_c07-543563.html
Cisco Nexus 5000 Series NX-0S FIbre Channel over Ethernet Configuration
Guide at: http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/fcoe/b
_Cisco_Nexus_5000_Series_NX-OS_Fibre_Channel_over_Ethernet_Confi guration_Guide_.html
.
Cisco Nexus 5000 Series CLI Reference Manual
SN0054671-00 D 2-37
