advertisement
Mellanox PreBoot Drivers
(FlexBoot & UEFI)
User Manual
Rev 3.0
www.mellanox.com Mellanox Technologies
NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (
“PRODUCT(S)”) AND ITS RELATED
DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY
KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT
USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST
ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY
QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES
CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE
HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR
ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE
PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE.
Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: (408) 970-3400
Fax: (408) 970-3403
© Copyright 2016. Mellanox Technologies Ltd. All Rights Reserved.
Mellanox®, Mellanox logo, Accelio®, BridgeX®, CloudX logo, CompustorX®, Connect-IB®, ConnectX®,
CoolBox®, CORE-Direct®, EZchip®, EZchip logo, EZappliance®, EZdesign®, EZdriver®, EZsystem®,
GPUDirect®, InfiniHost®, InfiniBridge®, InfiniScale®, Kotura®, Kotura logo, Mellanox CloudRack®, Mellanox
CloudXMellanox®, Mellanox Federal Systems®, Mellanox HostDirect®, Mellanox Multi-Host®, Mellanox Open
Ethernet®, Mellanox OpenCloud®, Mellanox OpenCloud Logo®, Mellanox PeerDirect®, Mellanox ScalableHPC®,
Mellanox StorageX®, Mellanox TuneX®, Mellanox Connect Accelerate Outperform logo, Mellanox Virtual Modular
Switch®, MetroDX®, MetroX®, MLNX-OS®, NP-1c®, NP-2®, NP-3®, Open Ethernet logo, PhyX®, PlatformX®,
PSIPHY®, SiPhy®, StoreX®, SwitchX®, Tilera®, Tilera logo, TestX®, TuneX®, The Generation of Open Ethernet logo, UFM®, Unbreakable Link®, Virtual Protocol Interconnect®, Voltaire® and Voltaire logo are registered trademarks of Mellanox Technologies, Ltd.
All other trademarks are property of their respective owners.
For the most updated list of Mellanox trademarks, visit http://www.mellanox.com/page/trademarks
Document Number: 3218 Mellanox Technologies 2
Table of Contents
Adding Device Driver Support in Linux Kernels/Initrd ............................................ 16
– Method I ................................................................ 16
– Method II – FlexBoot Only ..................................... 16
– Method III .............................................................. 17
Rev 3.0 Mellanox Technologies 3
4
Mellanox Technologies Rev 3.0
Rev 3.0 Mellanox Technologies 5
6
Configuring an iSCSI Target in Linux Environment ................................................ 61
CHAP Authentication and CHAP Mutual Authentication ........................................ 64
Mellanox Technologies Rev 3.0
List of Figures
Rev 3.0 Mellanox Technologies 7
List of Tables
8 Mellanox Technologies Rev 3.0
Document Revision History
Table 1: Document Revision History
Revision Date
3.0
Description
October 31 st
, 2017
Updated firmware versions, see section Supported Mellanox
Added the ESC option to the PXE boot, see section Skipping
FlexBoot, FlexBoot User Interface and FlexBoot Port
2.9
2.8
June 29
April 12 th th
, 2017
, 2017
Added the following sections:
Updated the following sections:
Burning the Expansion ROM Image (step 2)
FlexBoot System Settings Configuration Options
Updated firmware versions, see section Supported Mellanox
2.7
2.6
2.5
2.4
2.3
January 31 st
, 2017
Added the following sections:
September 29, 2016
Added the following sections:
Removed the following section:
Option ROM from both the FlexBoot and UEFI GUIs
May 31st, 2016
Added the following section:
January 31, 2016
Added the following sections:
HII System Settings Configuration Options
subsections
Booting from FlexBoot via IPv6
September 10, 2015
Added the following sections:
Updated the following sections:
Supported Mellanox Adapter Devices and Firmware
Rev 3.0 Mellanox Technologies 9
Revision Date
2.2
2.1
2.0
July 08, 2015
June 23, 2015
May 15, 2015
February 09, 2015
January 15, 2015
Description
Removed the following section:
FlexBoot Package
Updated the “Value restrictions” description in section Number of
Added the following sections:
CHAP Authentication and CHAP Mutual Authentication
Updated the following sections:
Removed the following sections:
Max Number of VFs Supported
Virtual MAC Address
Removed sections:
iSCSI General Parameters
DHCP Parameters
DHCP IP
DHCP Parameters
IP Version
Initial release of the restructured new User Manual.
10 Mellanox Technologies Rev 3.0
1
1.1
1.2
Mellanox PreBoot Drivers (FlexBoot & UEFI)
Overview
Mellanox PreBoot drivers are a multiprotocol remote boot technology that support remote
Boot over InfiniBand (BoIB) and over Ethernet.
Using Mellanox Virtual Protocol Interconnect (VPI) technologies available in ConnectX® adapters, PreBoot drivers give IT Managers’ the choice to boot from a remote storage target
(iSCSI target) or a LAN target (Ethernet Remote Boot Server) using a single ROM image on
Mellanox ConnectX products.
FlexBoot is based on the open source project iPXE available at http://www.ipxe.org.
PreBoot drivers first initialize the adapter device, senses the port protocol – Ethernet or
InfiniBand, and brings up the port. Then it connects to a DHCP server to obtain its assigned
IP address and network parameters, and also to obtain the source location of the kernel/OS to boot from. The DHCP server instructs the PreBoot drivers to access the kernel/OS through a
TFTP server, an iSCSI target, or some other service.
For an InfiniBand port, Mellanox PreBoot drivers implement a network driver with IP over
IB acting as the transport layer. IP over IB is part of the Mellanox OFED for Linux software package (see www.mellanox.com Products InfiniBand/VPI Drivers FlexBoot).
Supported Mellanox Adapter Devices and Firmware
The package supports the following HCA and firmware versions:
Table 2: Supported Mellanox Adapter Devices and Firmware
HCAs
ConnectX®-3
ConnectX®-3 Pro
ConnectX®-4
ConnectX®-4 Lx
ConnectX®-5
ConnectX®-5 Ex
Firmware Version FlexBoot Version
2.42.5000 3.4.752
2.42.5000 3.4.752
12.21.1000
14.21.1000
16.21.1000
16.21.1000
3.5.305
3.5.305
3.5.305
3.5.305
Rev 3.0 Mellanox Technologies 11
2 Burning the Expansion ROM Image
12
2.1
2.1.1
2.1.2
Burning the Image on ConnectX® Family Devices
Prerequisites
Expansion ROM Image
The expansion ROM images can be compiled using the Mellanox FlexBoot code source available in the FlexBoot webpage on www.mellanox.com
.
For further information on how to compile the code source, please refer to the pxebuild.py file inside the code source.
Firmware Burning Tools
You need to install the Mellanox Firmware Tools (MFT) package (version 4.1.0 or later) in order to burn the PXE ROM image. To download MFT, see Firmware Tools under www.mellanox.com > Products > InfiniBand/VPI Drivers > Firmware Tools.
Burning the Expansion ROM Image
NOTE: On selected platforms when burning the UEFI ROM image on a particular device, the UEFI controls other devices of the same type, if such devices do not have
UEFI burned on them.
To burn the composite image, perform the following steps:
1. Obtain the MST device name. Run:
# mst start
The device name will be of the form:
mt<dev_id>_pci{_cr0|conf0}
.
1
2. Create and burn the composite image. Run: flint -d <mst device name> --allow_rom_change brom <expansion ROM image path>
NOTE: If the following error “-E- Burn ROM failed: Unsupported operation under Controlled/Secure FW, please retry with -no_fw_ctrl” appears during the burning rom process, use the following burn command instead: flint -d <mst device name> --allow_rom_change --no_fw_ctrl brom <expansion ROM image path>
FlexBoot Example on Linux:
flint -d /dev/mst/mt4119_pci_cr0 --allow_rom_change brom FlexBoot-
3.5.110_4119.mrom
FlexBoot Example on Windows:
flint -d mt4119_pci_cr0 --allow_rom_change brom FlexBoot-
3.5.110_4119.mrom
1
Depending on the OS, the device name may be superseded with a prefix.
Mellanox Technologies Rev 3.0
2.2
UEFI Example on Linux:
flint -d /dev/mst/mt4119_pci_cr0 --allow_rom_change brom
14_12_24_RELEASE_0x1017.efirom
UEFI Example on Windows:
flint -d mt4119_pci_cr0 --allow_rom_change brom
14_12_24_RELEASE_0x1017.efirom
Removing the Expansion ROM Image
Remove the expansion ROM image. Run: flint -d <mst device name> drom
NOTE: When removing the expansion ROM image, you also remove PreBoot drivers from the boot device list.
Rev 3.0 Mellanox Technologies 13
3
3.1
3.1.1
Preparing the DHCP Server in Linux
When the boot session starts, the PXE firmware attempts to bring up an adapter network link (port). If it succeeds to bring up a connected link, the PXE firmware communicates with the DHCP server. The DHCP server assigns an IP address to the PXE client and provides it with the location of the boot program.
Case I: InfiniBand Ports
Installing DHCP
PreBoot drivers require that the DHCP server runs on a machine which supports IP over IB.
Prior to installing DHCP, make sure that Mellanox OFED for Linux is already installed on your DHCP server – see www.mellanox.com.
ISC DHCP of versions above 3.1.3 have native support for IPoIB clients. If using DHCP v3.1.3, the provided patch must be deployed. Newer versions do not require patching.
To download and install other supported DHCP server versions, please download
Mellanox OFED from www.mellanox.com Products InfiniBand/VPI Drivers Linux
SW/Drivers and refer to docs/dhcp/README.
Standard DHCP fields holding MAC addresses are not large enough to contain an IPoIB hardware address. To overcome this problem, DHCP over InfiniBand messages convey a client identifier field (in DHCP option 61) used to identify the DHCP session. This client identifier field can be used to associate an IP address with a client identifier value, such that the DHCP server will grant the same IP address to any client that conveys this client identifier.
NOTE: In the DHCP discover packets, Flexboot sends "hlen" 6 rather than 0.
"chaddr" will be the ethernet mac address rather than zeros.
In the DHCP server, the GUID in the client identifier can still be used to identify clients.
3.1.2
3.1.2.1
Configuring the DHCP Server
For ConnectX Family Devices
When a FlexBoot client boots, it sends the DHCP server various information including its DHCP client identifier. This identifier is used to distinguish between the various DHCP sessions. The value of the client identifier is composed of a prefix — ff:00:00:00:00:00:02:00:00:02:c9:00 — and an 8-byte port GUID (all separated by colons and represented in hexadecimal digits).
14 Mellanox Technologies Rev 3.0
3.1.2.2
3.1.2.3
Extracting the Port GUID – Method I
To obtain the port GUID:
1. Start mst. host1# mst start host1# mst status
2. The following MFT commands assume that the Mellanox Firmware Tools (MFT) package has been installed on the client machine.Obtain the Port GUID using the device name. The device name will be of the form: /dev/mst/ mt<dev_id>_pci{_cr0|conf0}. flint -d <MST_DEVICE_NAME> q
Assuming that FlexBoot is connected via Port 1, then the Port GUID is
00:02:c9:03:00:00:10:39
Extracting the Port GUID – Method II
An alternative method for obtaining the port GUID involves booting the client machine via
Flex- Boot. This requires having a Subnet Manager running on one of the machines in the
InfiniBand subnet. The 8 bytes can be captured from the boot session as shown in the figure below.
3.1.2.4
3.1.3
Placing Client Identifiers in /etc/dhcpd.conf
The following is an excerpt of a /etc/dhcpd.conf example file showing the format of representing a client machine for the DHCP server host host1 { next-server 11.4.3.7; filename "pxelinux.0"; fixed-address 11.4.3.130; option dhcp-client-identifier = ff:00:00:00:00:00:02:00:00:02:c9:00:00:02:c9:03:00:0c:78:11;
}
Running the DHCP Server
NOTE: This section is applicable only when the Initiator's IP address is empty.
In order for the DHCP server to provide configuration records for clients, an appropriate configuration file needs to be created. By default, the DHCP server looks for a configuration
Rev 3.0 Mellanox Technologies 15
3.1.4
file called dhcpd.conf under /etc . You can either edit this file or create a new one and provide its full path to the DHCP server using the -cf flag. See a file example at docs/dhcpd.conf of this package.
The DHCP server must run on a machine which has loaded the IPoIB module.
To run the DHCP server from the command line, enter:
dhcpd <IB network interface name> -d
Example:
host1# dhcpd ib0 –d
Adding Device Driver Support in Linux Kernels/Initrd
NOTE: A DHCP client can be used if you need to prepare a diskless machine with an IB
driver. See Step 11 under Example: Adding an IB Driver to initrd (Linux) .
3.2
3.2.1
Case II: Ethernet Ports
When a FlexBoot client boots, it sends the DHCP server various information, including its DHCP hardware Ethernet address (MAC). The MAC address is 6 bytes long, and it is used to distinguish between the various DHCP sessions.
Extracting the MAC Address – Method I
All Mellanox Ethernet NICs have a label on the printed side of the adapter card that has the card serial number and the card MAC address.
Figure 1: NIC Label
3.2.2 Extracting the MAC Address – Method II – FlexBoot Only
The six bytes of MAC address can be captured from the display upon the boot of the
ConnectX device session as shown in the figure below.
16 Mellanox Technologies Rev 3.0
3.2.3
3.2.4
Extracting the MAC Address – Method III
In case the previous methods fail to work, perform the following as a last resort:
1. Start mst. host1# mst start host1# mst status
The following MFT commands assume that the Mellanox Firmware Tools (MFT) package has been installed on the client machine.
2. Obtain the MAC using the device name. The device name will be of the form: /dev/mst/ mt<dev_id>_pci{_cr0|conf0}. flint -d <MST_DEVICE_NAME> q
Assuming that FlexBoot is connected via Port 1, then the MAC address is
00:02:c9:0c:7c:11.
Placing MAC Addresses in /etc/dhcpd.conf
The following is an excerpt of a /etc/dhcpd.conf example file showing the format of representing a client machine for the DHCP server running on a Linux machine. host host1 { next-server 11.4.3.7; filename "pxelinux.0"; fixed-address 11.4.3.130; hardware ethernet 00:02:c9:0c:78:12;
}
Rev 3.0 Mellanox Technologies 17
4 Subnet Manager – OpenSM
NOTE: This section applies to ports configured as InfiniBand only.
FlexBoot requires a Subnet Manager to be running on one of the machines in the IB network. OpenSM is part of the Mellanox OFED for Linux software package and can be used to accomplish this. Note that OpenSM may be run on the same host running the DHCP server but it is not mandatory.
For large InfiniBand clusters (> 100 nodes), it is recommended to use OpenSM caching. For further information, please refer to the Mellanox OFED User Manual.
18 Mellanox Technologies Rev 3.0
5
5.1
5.2
BIOS Configuration
FlexBoot (Legacy BIOS Mode)
The expansion ROM image presents itself to the BIOS as a boot device. As a result, the
BIOS will add to the list of boot devices “MLNX FlexBoot <ver>” for a ConnectX family device. The priority of this list can be modified through BIOS setup.
UEFI (UEFI BIOS Mode)
The expansion ROM image presents itself to the BIOS as a boot device. If BIOS is set to
UEFI Boot mode it will add to the list of boot devices “UEFI Network X [Device Name]” for each available port of ConnectX family devices. The priority of this list can be modified through BIOS setup.
Rev 3.0 Mellanox Technologies 19
6
6.1
6.2
Flexboot User Interface
NOTE: Boot menu is displyed as READ ONLY if the HCA card does not support flash configuration.
Skipping FlexBoot, FlexBoot User Interface and FlexBoot Port
You can skip FlexBoot in any of the given options below:
Following system POST, when a FlexBoot advertisement banner and configuration prompt is displayed, Press any key other than CTRL-B.
Skip a specific port when booting starts: Press CTRL-C or ESC
Invoking the User Interface
Shortly after completion of the POST (Power-On-Self-Test sequence of the computer), the user will be prompted to press CTRL-B to invoke Mellanox FlexBoot User Interface. The user has few seconds to press CTRL-B before the message disappears.
6.2.1 Browsing the User Interface Menu
To browse the menu, use the following keys:
Table 3: PXE Browsing Keys
Key
Arrows ↑↓
Enter
Esc
Operation
Browse options ( Highlighted option is current)
On forms – to enter menu
On configurable settings – edit configuration
Exit current menu
20 Mellanox Technologies Rev 3.0
6.2.2 Accessing FlexBoot Sub Menus
To enter the required menu, press Enter while the option is highlighted.
Figure 2: Main Menu Window
6.3
6.3.1
For further information on FlexBoot varies menus, and the configuration options they
provide, please refer to section Configuring System Settings and its subsections.
Configuring System Settings
NOTE: It is possible to save the changes made and exit the menu at any given time by pressing Ctrl+S.
Editing the Parameters using Free Text
When a configurable setting is highlighted, the instructions to edit it will be shown on screen.
If the setting is configurable via input, the description will show the input restrictions.
To change the configuration:
1. Press Enter or start typing while the setting is highlighted.
2. [Optional] To discard the input while editing, press Ctrl+C.
3. Press Enter and wait to verify the new value is accepted and shown.
In case the value entered is invalid, an alert will be shown and the new value won’t be accepted.
NOTE: Although numeric values are shown in decimal base, it is possible to insert values in different bases:
For hexadecimal base type “0x” before the number
For octal base type “0” before the number
For decimal base type the number as is
Rev 3.0 Mellanox Technologies 21
Figure 3: Free Text Editing Example
22
6.3.2
6.3.3
6.3.4
Editing the Parameters using Set Values
When a configurable setting is highlighted, the instructions to edit it will be shown on screen.
If the setting is configurable with built-in options, the description will be “Select to change value”.
To change the configuration:
1. Press Enter while setting is highlighted.
2. Keep pressing Enter to choose the desired value
Deleting Configurable Settings
Configurable settings can be deleted. Value will only be deleted from the current configuration and not from the flash. If the configuration has a default value, it will be set when deleting the current value.
To delete a configurable setting:
1. Press Ctrl+D while setting is highlighted.
Restoring Default Configurations
All configurations can be restored to the default values set in the device. All configurable values will be modified to “default” or “deleted” if there is no default value in the device.
To restore defaults per port:
Press Ctrl+R on the main menu of the port only when applicable
To restore defaults per device:
Press Ctrl+R on “System setup” menu when applicable
Mellanox Technologies Rev 3.0
6.4 Exiting FlexBoot User Interface
To exit FlexBoot User Interface press ESC from the System Setup menu.
Figure 4: Exit Menu
Rev 3.0 Mellanox Technologies 23
7
7.1
FlexBoot System Settings Configuration Options
Below are listed the attributes shown in varies FlexBoot User Interface menus.
System Setup
In the following menu, you can set the following options:
[ConnectX-5 Only] Power Configuration
Figure 5: System Setup Menu
24 Mellanox Technologies Rev 3.0
7.1.1 Diagnostic
Diagnostic menu enables the user to diagnose problems in released ROMs by enabling the debug log levels for specific modules. This ability should be used only when debug session is needed.
Figure 6: Diagnostic Properties Menus
7.1.1.1 boot_to_shell
Location:
Description:
Configurable:
Available options:
Diagnostic
When set to enable, Flexboot well boot to ipxe shell instead of the normal boot.
Yes
1 - Enable
2 - Disable
Rev 3.0 Mellanox Technologies 25
7.1.1.2
7.1.2
7.1.3
7.1.4
7.1.5
7.1.6
Persistency: panic_behavior
Location:
Description:
Configurable:
Available options:
Persistency:
Persistent through AC cycle
Diagnostic
Flexboot behavior on critical failure.
when set to PRINT , print error to screen
when set to HALT - freeze system
when set to OPEN_SHELL, open ipxe shell
when set to REG_DUMP dump FW registers and exit to shell
Yes
1 - PRINT
2 - HALT
3 - OPENSHELL
4 - REG_DUMP
Persistent through AC cycle
Bus:Device:Function
Location:
Description:
Configurable:
System setup
Specifies the BIOS assigned PCI Bus:Device:Function identifier of the card
No
Chip Type
Location:
Description:
Configurable:
Device Name
Location:
Description:
Configurable:
System setup
Specifies the chip type
No
System setup
The device’s product name
No
PCI Device ID
Location:
Description:
Configurable:
System setup
The PCI Device ID of the controller
No
Banner Menu Timeout
Location:
System setup
26 Mellanox Technologies Rev 3.0
7.1.7
7.1.8
Description:
Configurable:
Persistency:
Reboot required:
Available options:
Defines the Banner's timeout
Yes – via input
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
1-14 sec
Virtualization Mode
Location:
Description:
Configurable:
Persistency:
Reboot required:
Available options:
System setup
Specifies the type of virtualization used by the controller on all ports.
Only valid for adapters that support SR-IOV virtualization mode.
Yes – built-in options
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
1. None
2. SR-IOV
Number of Virtual Functions
Location:
Description:
Configurable:
Persistency:
Reboot required:
Value restrictions:
System setup
The number of virtual functions advertised and usable by the driver. Only valid for adapters that support SR-IOV virtualization mode.
Yes – via input – If virtualization mode is SR-IOV
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
Minimum integer value is 0. Maximum integer value is the value supported by the firmware installed.
Rev 3.0 Mellanox Technologies 27
7.2 Power Configuration
[ConnectX-5 Only] In the following menu, you can set advanced power settings:
Figure 7: Power Configuration Properties Menu
7.2.1
7.2.2
Advanced Power Settings
Location:
Description:
Configurable:
Available options:
Power Configuration
When set to Enabled, additional power settings parameter are configurable.
Yes
1. Enabled
2. Disabled
Disable Slot Power Limiter
Location:
Description:
Configurable:
Available options:
Power Configuration
When set, the slot power limiter is disabled, and the device is allowed to consume more than 25W from the PCIe power rails.
Yes
1. Enabled
2. Disabled
28 Mellanox Technologies Rev 3.0
7.3 Firmware Image Properties
In the following menu, you can set the following options:
Figure 8: Firmware Image Properties Menu
7.3.1
7.3.2
Flexboot Version
Location:
Description:
Configurable:
Available options:
System setup /Firmware Image Properties
Flexboot driver version information
No
3. None – no PXE boot, no iSCSI boot
4. PXE – PXE first and if failed try iSCSI iSCSI – iSCSI first and if failed try PXE
Family Firmware Version
Location:
Description:
Configurable:
System setup /Firmware Image Properties
Device’s firmware version information.
No
Rev 3.0 Mellanox Technologies 29
7.4 Main Configuration
In the following menu, you can set the following options:
Figure 9: Main Configuration
30
7.4.1 Blink LEDs
NOTE: Blink LEDs will not appear in the boot menu if the feature is not supported by the firmware.
NOTE: LEDs starts to blink as soon as you press the Enter key.
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations
Specifies the number of seconds the LEDs on physical network port should blink to assist with port identification. Only valid for adapters with blink LEDs.
Yes – via input
Valid only for current cycle
Blink LEDs duration value of the port will be as set in the current cycle
The minimum integer value is 0 and the maximum is 15.
Mellanox Technologies Rev 3.0
7.4.2
7.5
7.5.1
MAC Address
Location:
Description:
Configurable:
System setup/Port main configurations
Permanent MAC address assigned during manufacturing
No
NIC Configuration
Ethernet NIC Configuration
In the following menu, you can set the following options:
Figure 10: Ethernet NIC Configuration
7.5.1.1 Legacy Boot Protocol
Location:
Description:
System setup/Port main configurations/NIC Configuration
Use a non-UEFI network boot protocol.
The chosen protocol is tried first; if boot fails the other protocol may be used.
Rev 3.0 Mellanox Technologies 31
32
7.5.1.2
7.5.1.3
7.5.1.4
Configurable:
Persistency:
Available options:
Notes:
1. If set to NONE, the boot will be skipped on this port.
2. IF all the ports per Bus Device function (BDF) are set as NONE, a
BEV entry is not registered.
Yes – built-in options
Persistent through AC cycle
5. None – no PXE boot, no iSCSI boot
6. PXE – PXE first and if failed try iSCSI
7. iSCSI – iSCSI first and if failed try PXE
IPv4/IPv6 support
Location:
Description:
Configurable:
Persistency:
Reboot required:
Available options:
System setup/Port main configurations/NIC Configuration
Select which IP protocol version will be used when booting. If both are configured, FlexBoot will try to boot with the 2 nd
protocol only if failed to boot with the 1 st
one.
Yes – built-in option
Persistent through AC cycle
No
1. IPv4
2. IPv6
3. IPv4/IPv6
Boot Retry Count
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/NIC Configuration
The number of retries to attempt in case of a boot failure.
Yes – built-in options
Persistent through AC cycle
0. No retries
1. Retry
2. Retries
3. Retries
4. Retries
5. Retries
6. Retries
7. Indefinite Retries
Undi network wait timeout
Location:
Description:
Configurable:
Value restrictions:
System setup/Port main configurations/NIC Configuration
When the UNDI interface is used, defines the delay time when
PXENV_UNDI_OPEN call is issued.
Yes – via input
The minimum integer value is 0 and Maximum integer value is 30.
Mellanox Technologies Rev 3.0
7.5.1.5
7.5.1.6
7.5.1.7
7.5.1.8
Virtual LAN ID
Location:
Description:
Configurable:
Value restrictions:
Virtual LAN Mode
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/NIC Configuration
Uses the VLAN tag with the selected boot protocol.
Yes – via input – If virtual LAN mode is enabled
The minimum integer value is 1 and Maximum integer value is 4094.
System setup/Port main configurations/NIC Configuration
Enable/Disable VLAN tagging on the selected boot protocol.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
VPI link type
Location:
Description:
Configurable:
Persistency:
Reboot required:
Available options:
System setup/Port main configurations/NIC Configuration
Specifies the network link type for supported cards.
Note: This option is available only on supported devices.
Yes – built-in option
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
1. InfiniBand
2. Ethernet
Wake on LAN
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/NIC Configuration
Status of Wake on LAN feature. When set, it enables the server to be powered using an in-band magic packet.
Only valid for adapters that support Wake on LAN.
Note: This option is available only on supported devices.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
Rev 3.0 Mellanox Technologies 33
7.5.2 InfiniBand NIC Configuration
In the following menu, you can set the following options:
Figure 11: InfiniBand NIC Configuration
7.5.2.1 PKey Value
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/NIC Configuration
PKey value to be used when booting IPoIB
Yes
Persistent through AC cycle
0-65535
0 – the default PKey value
34 Mellanox Technologies Rev 3.0
7.6 iSCSI General Parameters
In the following menu, you can set the following options:
Figure 12: iSCSI General Parameters
7.6.1
7.6.2
DHCP IP
Location:
Description:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
When the DHCP IP is enabled, the initiator and target’s IP, netmask, gateway and DNS will be taken from DHCP instead of from the menu.
Yes
Configurable:
DHCP Parameters
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
When the DHCP IP and DHCP Parameters are enabled, all the initiator and target’s parameters are taken from DHCP instead of from the menu.
Yes
Rev 3.0 Mellanox Technologies 35
7.6.3
7.6.4
7.6.5
7.6.6
IP Version
Location:
Description:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Specifies whether IPv4 or IPv6 network addressing will be used for iSCSI initiator and targets
No
Configurable:
CHAP Authentication
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Enables CHAP Authentication for iSCSI initiator.
Yes
CHAP Mutual Authentication
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Enables mutual CHAP authentication between the iSCSI initiator and target
Yes
Boot to target
Location:
Description:
Configurable:
Available options:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Controls booting to ISCSI target after connection.
Yes
1. Enable
2. Disable
3. One time disabled
36 Mellanox Technologies Rev 3.0
7.7 iSCSI Initiator Parameters
NOTE: iSCSI Initiator Parameters menu is applicable to ConnectX®-3/ConnectX®-3
Pro adapter cards only.
In the following menu, you can set the following options:
Figure 13: iSCSI Initiator Parameters
7.7.1 CHAP ID
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Challenge-Handshake Authentication Protocol
(CHAP) ID
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Mutual Authentication are enabled – This will be the reverse username used for the iSCSI initiator Challenge-
Handshake Authentication Protocol.
Rev 3.0 Mellanox Technologies 37
7.7.2
7.7.3
7.7.4
7.7.5
Value restrictions:
CHAP Secret
Location:
Maximum string length is 128.
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Challenge-Handshake Authentication Protocol
(CHAP) secret (password).
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Mutual Authentication are enabled – This will be the reverse password used for the iSCSI initiator Challenge-
Handshake Authentication Protocol.
String length should be 0 or 12 to 16
Value restrictions:
IPv4 Address
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters iSCSI initiator’s IPv4 address
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the IP address of the initiator.
Should be in IPv4 format
IPv4 Primary DNS
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Primary DNS IP Address.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the DNS for the
IP address of the initiator.
Should be in IPv4 format
Value restrictions:
IPv4 Default Gateway
Location:
Description:
Configurable:
Persistency:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator default Gateway IP address.
Yes – via input
Persistent through AC cycle
38 Mellanox Technologies Rev 3.0
7.7.6
7.7.7
Effects:
If the legacy boot protocol is iSCSI – This will be set as the default gateway IP address of the initiator.
Should be in IPv4 format
Value restrictions: iSCSI Name
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The initiator's iSCSI Qualified Name (IQN).
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the iSCSI
Qualified Name of the initiator.
Maximum string length is 223
Value restrictions:
Subnet Mask
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Subnet Mask for an IPv4 initiator configuration.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the subnet mask for the IP address of the initiator.
Should be in IPv4 format
Rev 3.0 Mellanox Technologies 39
7.8 iSCSI First Target Parameters
NOTE: iSCSI First Target Parameters menu is applicable to ConnectX®-3/ConnectX®-3 Pro adapter cards only.
In the following menu, you can set the following options:
7.8.1 Boot LUN
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The boot Logical Unit Number (LUN) on the first iSCSI storage target.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the boot LUN in the root path for the first target.
The minimum integer value is 0 and the maximum is 256.
40 Mellanox Technologies Rev 3.0
7.8.2
7.8.3
7.8.4
7.8.5
Connect
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
Enable/Disable connecting to the first iSCSI target. This will be ignored if both DHCP IP and DHCP Parameters are enabled in iSCSI general parameters.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
iSCSI Name
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
CHAP ID
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The iSCSI Qualified Name (IQN) of the first target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the iSCSI Qualified name in the root path for the first target.
Maximum string length is 223
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The first iSCSI storage target Challenge-Handshake Authentication
Protocol (CHAP) ID.
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Authentication are enabled – This will be the username used for the first target Challenge-Handshake
Authentication Protocol.
Maximum string length is 128.
Value restrictions:
CHAP Secret
Location:
Description:
Configurable:
Persistency:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The Challenge-Handshake Authentication Protocol secret (CHAP password) of the first iSCSI storage target.
Yes – via input
Persistent through AC cycle
Rev 3.0 Mellanox Technologies 41
7.8.6
7.8.7
Effects:
If the Connect and the CHAP Authentication are enabled – This will be the password used for the first target Challenge-Handshake
Authentication Protocol.
String length should be 0 or 12 to 16
Value restrictions:
IP Address
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The IP address of the first iSCSI target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the IP address in the root path for the first target.
Should be in IPv4 format
Value restrictions:
TCP Port
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
TCP Port number of first iSCSI target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the TCP Port in the root path for the first target.
The minimum integer value is 1, and the maximum is 65535.
42 Mellanox Technologies Rev 3.0
8
8.1
HII System Settings Configuration Options
Below are listed the attributes shown in varies HII menus.
Main Configuration Page
In the following menu, you can see the following options:
Figure 14: Main Configuration Form
8.1.1 Blink LEDs
NOTE: Blink LEDs will not appear in the boot menu if the feature is not supported by the firmware.
Location:
Description:
Configurable:
Rev 3.0 Mellanox Technologies
Main Configuration Form
Specifies the number of seconds the LEDs on physical network port should blink to assist with port identification. Only valid for adapters with blink LEDs.
Yes – via input
43
44
8.1.2
8.1.3
8.1.4
8.1.5
8.1.6
8.1.7
Persistency:
Reboot required:
Available options:
Device Name
Location:
Description:
Configurable:
Valid only for current cycle
No
0-15 sec
Main Configuration Form
The device’s product name
No
Chip Type
Location:
Description:
Configurable:
PCI Device ID
Location:
Description:
Configurable:
PCI Address
Location:
Description:
Main Configuration Form
Specifies the chip type
No
Main Configuration Form
The PCI Device ID of the controller
No
Main Configuration Form
Specifies the BIOS assigned PCI Bus:Device:Function identifier of the card
No
Configurable:
Link Status
Location:
Description:
Configurable:
Main Configuration Form
Displays current link status on port
No
Network Link Type
NOTE: Network Link Type will not appear in the boot menu if the feature is not supported by the firmware.
Location:
Description:
Configurable:
Main Configuration Form
Allows user to manually choose link type on selected VPI devices
Yes – via input
Mellanox Technologies Rev 3.0
8.1.8
8.1.9
8.2
Persistency:
Reboot required:
Available options:
MAC Address
Location:
Description:
Configurable:
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
Ethernet/InfiniBand/VPI
Main Configuration Form
Permanent MAC address assigned during manufacturing
No
Virtual MAC Address
Location:
Description:
Configurable:
Main Configuration Form
Displays device’s virtual MAC address
No
Firmware Image Properties
In the following menu, you can set the following options:
Figure 15: Firmware Image Properties Menu
Rev 3.0 Mellanox Technologies 45
8.2.1
8.2.2
8.3
Family Firmware Version
Location:
Description:
Configurable:
Main Configuration Form /Firmware Image Properties
Device’s firmware version information.
No
EFI Version
Location:
Description:
Configurable:
Main Configuration Form /Firmware Image Properties
Device’s UEFI driver version information.
No
NIC Configuration
In the following menu, you can set the following options:
Figure 16: Ethernet NIC Configuration
46
8.3.1 Banner Message Timeout
Location:
Description:
Configurable:
Main Configuration Form /NIC Configuration
Defines the Banner's timeout
Yes – via input – If virtualization mode is SR-IOV
Mellanox Technologies Rev 3.0
8.3.2
8.3.3
8.3.4
8.3.5
Persistency:
Reboot required:
Available options:
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
0-14 sec
Legacy Boot Protocol
Location:
Description:
Configurable:
Persistency:
Available options:
Main Configuration Form /NIC Configuration
Use a non-UEFI network boot protocol.
The chosen protocol is tried first; if boot fails the other protocol will be used.
Yes – built-in options
Persistent through AC cycle
4. None – no PXE boot, no iSCSI boot
5. PXE – PXE first and if failed try iSCSI
6. iSCSI – iSCSI first and if failed try PXE
Virtual LAN Mode
Location:
Description:
Configurable:
Persistency:
Available options:
Main Configuration Form /NIC Configuration
Enable/Disable VLAN tagging on the selected boot protocol.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
Virtual LAN ID
Location:
Description:
Configurable:
Value restrictions:
PKey
Main Configuration Form /NIC Configuration
Uses the VLAN tag with the selected boot protocol.
Yes – via input – If virtual LAN mode is enabled
The minimum integer value is 1 and Maximum integer value is 4094.
NOTE: PKey setting will appear in the boot menu only for selected adapters that support the feature and only in InfiniBand mode.
Location:
Description:
Configurable:
Value restrictions:
Main Configuration Form /NIC Configuration
PKey ID to be used by PXE boot. If 0 is selected, the default PKey will be used.
Yes – via input – If PKey is supported
The minimum integer value is 0 and Maximum integer value is 65535.
Rev 3.0 Mellanox Technologies 47
8.3.6
8.3.7
8.3.8
Link Speed
Location:
Description:
Configurable:
Value restrictions:
Boot Retry Count
Location:
Description:
Configurable:
Persistency:
Available options:
Main Configuration Form /NIC Configuration
Defines link working speed
Currently no, only exposes Auto Negotiated state.
-
Main Configuration Form /NIC Configuration
The number of retries to attempt in case of a boot failure.
Yes – built-in options
Persistent through AC cycle
0. No retries
1. Retry
2. Retries
3. Retries
4. Retries
5. Retries
6. Retries
7. Indefinite Retries
Wake on LAN
Location:
Description:
Configurable:
Persistency:
Available options:
Main Configuration Form /NIC Configuration
Status of Wake on LAN feature. When set, it enables the server to be powered using an in-band magic packet.
Only valid for adapters that support Wake on LAN.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
48 Mellanox Technologies Rev 3.0
8.4 Device Level Configuration
In the following menu, you can set the following options:
PCI Virtual Functions Advertised
Figure 17: Device Level Configuration
8.4.1
8.4.2
Virtualization Mode
Location:
Description:
Configurable:
Persistency:
Reboot required:
Available options:
Main Configuration Form /Device Level Configuration
Specifies the type of virtualization used by the controller on all ports.
Only valid for adapters that support SR-IOV virtualization mode.
Yes – built-in options
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
7. None
8. SR-IOV
PCI Virtual Functions Advertised
Location:
Description:
Configurable:
Persistency:
Reboot required:
Value restrictions:
Main Configuration Form /Device Level Configuration
The number of virtual functions advertised and usable by the driver. Only valid for adapters that support SR-IOV virtualization mode.
Yes – via input – If virtualization mode is SR-IOV
Persistent through AC cycle
Yes – reboot is needed in order for this change to apply
Minimum integer value is 0. Maximum integer value is the value supported by the firmware installed.
Rev 3.0 Mellanox Technologies 49
8.5 iSCSI General Parameters
In the following menu, you can set the following options:
Figure 18: iSCSI General Parameters
50
8.5.1
8.5.2
Boot to Target
Location:
Description:
Configurable:
Available options:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Controls booting to ISCSI target after connection.
Yes
Enable
Disable
One time disabled
TCP/IP Parameters via DHCP
Location:
Description:
Configurable:
Main Configuration Form /iSCSI Configuration/iSCSI General
Parameters
When the TCP/IP Parameters via DHCP is enabled, the initiator and target’s IP, netmask, gateway and DNS will be taken from DHCP instead of from the menu.
Yes
Mellanox Technologies Rev 3.0
8.5.3
8.5.4
8.5.5
8.5.6 iSCSI Parameters via DHCP
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
When iSCSI Parameters via DHCP are enabled, all the initiator and target’s parameters are taken from DHCP instead of from the menu.
Yes, if TCP/IP Parameters via DHCP is enabled.
CHAP Authentication
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Enables CHAP Authentication for iSCSI initiator.
Yes
CHAP Mutual Authentication
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Enables mutual CHAP authentication between the iSCSI initiator and target
Yes
IP Version
Location:
Description:
Configurable:
System setup/Port main configurations/iSCSI Configuration/iSCSI
General Parameters
Specifies whether IPv4 or IPv6 network addressing will be used for iSCSI initiator and targets
No
Rev 3.0 Mellanox Technologies 51
8.6 iSCSI Initiator Parameters
In the following menu, you can set the following options:
Figure 19: iSCSI Initiator Parameters
52
8.6.1
8.6.2
CHAP ID
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
CHAP Secret
Location:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Challenge-Handshake Authentication Protocol
(CHAP) ID
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Mutual Authentication are enabled – This will be the reverse username used for the iSCSI initiator Challenge-
Handshake Authentication Protocol.
Maximum string length is 128.
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
Mellanox Technologies Rev 3.0
8.6.3
8.6.4
8.6.5
Description:
Configurable:
Persistency:
Effects:
The iSCSI initiator Challenge-Handshake Authentication Protocol
(CHAP) secret (password).
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Mutual Authentication are enabled – This will be the reverse password used for the iSCSI initiator Challenge-
Handshake Authentication Protocol.
String length should be 0 or 12 to 16
Value restrictions:
IPv4 Address
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters iSCSI initiator’s IPv4 address
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the IP address of the initiator.
Should be in IPv4 format
Value restrictions:
IPv4 Primary DNS
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Primary DNS IP Address.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the DNS for the
IP address of the initiator.
Should be in IPv4 format
IPv4 Default Gateway
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator default Gateway IP address.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the default gateway IP address of the initiator.
Should be in IPv4 format
Rev 3.0 Mellanox Technologies 53
8.6.6
8.6.7 iSCSI Name
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The initiator's iSCSI Qualified Name (IQN).
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the iSCSI
Qualified Name of the initiator.
Maximum string length is 223
Value restrictions:
Subnet Mask
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI
Initiator Parameters
The iSCSI initiator Subnet Mask for an IPv4 initiator configuration.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI – This will be set as the subnet mask for the IP address of the initiator.
Should be in IPv4 format
54 Mellanox Technologies Rev 3.0
8.7 iSCSI First Target Parameters
In the following menu, you can set the following options:
Figure 20: iSCSI First Target Parameters
8.7.1 Boot LUN
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The boot Logical Unit Number (LUN) on the first iSCSI storage target.
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the boot LUN in the root path for the first target.
The minimum integer value is 0 and the maximum is
18446744073709551615.
Rev 3.0 Mellanox Technologies 55
8.7.2
8.7.3
8.7.4
8.7.5
Connect
Location:
Description:
Configurable:
Persistency:
Available options:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
Enable/Disable connecting to the first iSCSI target. This will be ignored if both DHCP IP and DHCP Parameters are enabled in iSCSI general parameters.
Yes – built-in options
Persistent through AC cycle
1. Enabled
2. Disabled
iSCSI Name
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
CHAP ID
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The iSCSI Qualified Name (IQN) of the first target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the iSCSI Qualified name in the root path for the first target.
Maximum string length is 223
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The first iSCSI storage target Challenge-Handshake Authentication
Protocol (CHAP) ID.
Yes – via input
Persistent through AC cycle
If the Connect and the CHAP Authentication are enabled – This will be the username used for the first target Challenge-Handshake
Authentication Protocol.
Maximum string length is 128.
Value restrictions:
CHAP Secret
Location:
Description:
Configurable:
Persistency:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The Challenge-Handshake Authentication Protocol secret (CHAP password) of the first iSCSI storage target.
Yes – via input
Persistent through AC cycle
56 Mellanox Technologies Rev 3.0
8.7.6
8.7.7
Effects:
If the Connect and the CHAP Authentication are enabled – This will be the password used for the first target Challenge-Handshake
Authentication Protocol.
String length should be 0 or 12 to 16
Value restrictions:
IP Address
Location:
Description:
Configurable:
Persistency:
Effects:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
The IP address of the first iSCSI target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the IP address in the root path for the first target.
Should be in IPv4 format
Value restrictions:
TCP Port
Location:
Description:
Configurable:
Persistency:
Effects:
Value restrictions:
System setup/Port main configurations/iSCSI Configuration/iSCSI First target Parameters
TCP Port number of first iSCSI target
Yes – via input
Persistent through AC cycle
If the legacy boot protocol is iSCSI and Connect is enabled – This will be the TCP Port in the root path for the first target.
The minimum integer value is 1, and the maximum is 65535.
Rev 3.0 Mellanox Technologies 57
8.8 Power Configuration
[ConnectX-5 Only] In the following menu, you can set advanced power settings:
Figure 21: Power Configuration Properties Menu
58
8.8.1
8.8.2
Advanced Power Settings
Location:
Description:
Configurable:
Available options:
Power Configuration
When set to Enabled, additional power settings parameter are configurable.
Yes
1. Enabled
2. Disabled
Disable Slot Power Limiter
Location:
Description:
Configurable:
Available options:
Power Configuration
When set, the slot power limiter is disabled, and the device is allowed to consume more than 25W from the PCIe power rails.
Yes
1. Enabled
2. Disabled
Mellanox Technologies Rev 3.0
9
9.1
9.1.1
9.1.2
How to Boot
PXE Boot
Prerequisites
Make sure that your client is connected physically and logically to the server(s)
The PreBoot drivers images are already programmed on the adapter card
For InfiniBand ports only: Start the Subnet Manager
Configure and start the DHCP server
Configure and start at least one of the services such as: iSCSI, tftp, http
Starting Boot
Boot the client machine and enter BIOS setup to configure “MLNX FlexBoot” to be the first on the boot device priority list.
NOTE: On dual-port ConnectX-3 network adapters, the client first attempts to boot from Port 1. If this fails, it switches to boot from Port 2. Note also that the driver waits up to 10 seconds for port come up and up to 90 seconds to assign an IP from the DHCP server.
If a PreBoot driver was selected through BIOS setup, the client will boot from FlexBoot. The client will display FlexBoot attributes, sense the port protocol (In ConnectX-3 cards only) –
Ethernet or InfiniBand. The port sensing occurs only when the port is connected to a QSFP connector. In case of an InfiniBand port, the client will also wait for port configuration by the Subnet Manager. If auto-sensing protocol fails, the port will be configured as an
InfiniBand port.
After configuring the IB/ETH port, the client attempts to connect to the DHCP server to obtain an IP address and the source location of the kernel/OS to boot from.
For ConnectX (InfiniBand):
Next, FlexBoot attempts to boot as directed by the DHCP server.
Rev 3.0 Mellanox Technologies 59
9.1.3
9.1.4
9.2
9.2.1
How to Boot to ESXi
For instructions on booting to ESXi, refer to the ESXi Installation Guide on VMWare site: http://pubs.vmware.com/vsphere-55/index.jsp#com.vmware.vsphere.upgrade.doc/GUID-
B9DB94CA-4857-458B-B6F1-6A688726AED0.html
How to Boot Using HTTP and iPXE Script
After performing basic HTTP configuration, add the following lines to the dhcp server configuration file: host uefi24-1 { hardware ethernet F4:52:14:7A:56:F1 ; fixed-address 14.7.6.24; filename "pxe_script_linux"; option vendor-class-identifier "PXEClient";
}
“pxe_script_linux” file:
#
!ipxe
dhcp net0 kernel -n vmlinuz http://14.7.6.30/RHEL6.4-x86_64-
DVD1/images/pxeboot/vmlinuz initrd http://14.7.6.30/RHEL6.4-x86_64-DVD1/images/pxeboot/initrd.img boot
iSCSI
Mellanox FlexBoot enables an iSCSI-boot of an OS located on a remote iSCSI target. It has a built-in iSCSI initiator which can connect to the remote iSCSI target and load from it the kernel and initrd. There are two instances of connection to the remote iSCSI target: the first is for get- ting the kernel and initrd via FlexBoot, and the second is for loading other parts of the OS via initrd.
If you choose to continue loading the OS (after boot) through the HCA device driver, please verify that the initrd image includes the HCA driver.
NOTE: To boot via iSCSI, Flexboot must be selected from the BIOS boot menu (it will be shown as a BEV entry). Do not look for the entry in the BVC table.
Installing to iSCSI Target
It is recommended to install to an iSCSI target using the same port performing the following:
1. Connect to the iSCSI target to retrieve various information from it.
Due to the step above, FlexBoot will register the remote target by hooking BIOS interrupt
13.
2. Download the installation image using the same port by supplying the file name in the client’s DHCP configuration file along with the root path.
60 Mellanox Technologies Rev 3.0
9.2.2
9.2.2.1
9.2.2.2
Configuring an iSCSI Target in Linux Environment
Prerequisites
Make sure that an iSCSI Target is installed on your server side.
You can download and install an iSCSI Target from the following location: http://sourceforge.net/projects/iscsitarget/files/iscsitarget/
Dedicate a partition on your iSCSI Target on which you will later install the operating system
Configure your iSCSI Target to work with the partition you dedicated. If, for example, you choose partition /dev/sda5, then edit the iSCSI Target configuration file /etc/ietd.conf to include the following line under the iSCSI Target iqn line:
Lun 0 Path=/dev/sda5,Type=fileio
Example of an iSCSI Target iqn line:
Target iqn.2007-08.7.3.4.10:iscsiboot
Start your iSCSI Target.
Example: host1# /etc/init.d/iscsitarget start
Configuring the DHCP Server to Boot From an iSCSI Target
1. Configure DHCP as described in section “ Preparing the DHCP Server in Linux
2. Edit your DHCP configuration file (/etc/dhcpd.conf) and add the following lines for the machine(s) you wish to boot from the iSCSI target:
Filename ""; option root-path "iscsi:iscsi_target_ip::::iscsi_target_iqn";
The following is an example for configuring an IB/ETH device to boot from an iSCSI target: host host1{ filename ""; option root-path "iscsi:iscsi_target_ip::::iscsi_target_iqn";
# For a ConnectX device with ports configured as InfiniBand, comment out# the follow- ing line
# option dhcp-client-identifier = ff:00:00:00:00:00:02:00:00:02:c9:00:00:02:c9:03:00:00:10:39;
# For a ConnectX device with ports configured as Ethernet, comment out# the following line
# hardware ethernet 00:02:c9:00:00:bb;
}
Rev 3.0 Mellanox Technologies 61
9.3 Booting from FlexBoot via IPv6
IPv6 boot must be enable it in FlexBoot menu. You can configure it to:
IPv4 only
IPv6 only
IPv4 configuration and then IPv6
62
To boot from FlexBoot via IPv6:
1. Run the RADVD service on a server in the network (preferably the DHCP server) and configure it to send a periodic Router Advertisement.
Example: interface eth1
{
AdvSendAdvert on;
MinRtrAdvInterval 3;
MaxRtrAdvInterval 5;
AdvManagedFlag on;
AdvOtherConfigFlag off;
AdvHomeAgentFlag off;
prefix fdfc::/64 {
AdvOnLink on;
AdvAutonomous off;
AdvRouterAddr off;
};
};
2. Configure the DHCP service.
Example: default-lease-time 100; preferred-lifetime 60; option dhcp-renewal-time 30; allow leasequery; option dhcp6.preference 255; option dhcp6.rapid-commit; option dhcp6.bootfile6-url code 59 = string; subnet6 fdfc::/64 {
option dhcp6.bootfile-url "tftp://[fdfc::123]/ipv6_script";
Mellanox Technologies Rev 3.0
9.4
9.4.1
option dhcp6.name-servers fdfc::123;
range6 fdfc::140 fdfc::200;
range6 fdfc:: temporary;
}
While the ipxe_script contents is:
#!ipxe
#echo "Attempting to download linux OS over IPv6..." kernel tftp://[fdfc::123]/kernel/vmlinuz_new initrd tftp://[fdfc::123]/kernel/ramdisk_new.gz imgselect vmlinuz_new echo "Booting..." boot or over HTTP:
#!ipxe
#echo "Attempting to download linux OS over IPv6..." kernel http://[fdfc::123]/kernel/vmlinuz_new initrd http://[fdfc::123]/kernel/ramdisk_new.gz imgselect vmlinuz_new echo "Booting..." boot
Legacy Boot Priority Behavior
The sections below describe various legacy boot priority behaviors based on the protocol used.
Legacy Boot Protocol PXE
The client first tries to perform PXE boot and if that fails, it tries iSCSI boot. The client goes to the DHCP to get an IP address. The DHCP server might supply filename (for PXE boot) and root-path (for iSCSI boot) if set by the user.
If the iSCSI target parameters are taken from the flash, then the root-path retrieved from the DHCP is deleted, and if the ‘Connect’ is set as “Enable” then the values stored in the flash are used (DHCP IP = Enable, DHCP Parameters = Disable).
If a root-path is supplied from either the DHCP or the flash, the client is connected to the iSCSI target but no boot is performed.
If a filename is supplied, a PXE boot is performed on the Operating System.
If PXE boot failed or no filename was supplied, an iSCSI boot is performed to the connected target.
Rev 3.0 Mellanox Technologies 63
9.4.2
9.4.3
Legacy Boot Protocol iSCSI
The client first tries to perform iSCSI boot and if that fails, it tries PXE boot. Upon iSCSI boot one of the following flows occurs:
If all iSCSI configuration is taken from the flash, the client waits 25 seconds for the switches to be configured, and then tries to perform an iSCSI boot (hook to target and then boot). If the boot fails, the client exits. (DHCP IP = disable)
Or
1. If only the TCP parameters need to be retrieved from the DHCP (DHCP IP = enable,
DHCP Parameters = disable), then the IP address is taken from the DHCP, the root-path retrieved from the DHCP is deleted and if the ‘Connect’ is set as “Enable” then the values stored in the flash are used.
2. Hook to the iSCSI target.
3. Try to boot to the iSCSI target.
4. If iSCSI boot failed and DHCP supplied a filename, try to PXE boot.
5. If PXE boot failed, exit.
When the legacy boot protocol iSCSI is selected, the following is not supported:
PXE boot after iSCSI boot with static configurations – Setting all iSCSI parameters from the flash (including IP address)
CHAP Authentication and CHAP Mutual Authentication
iSCSI boot allows CHAP security option if the following conditions are met:
If ‘Connect’ = disable or ‘CHAP Authentication’ = disable, the server ignores both the
CHAP Authentication and CHAP Mutual Authentication
If ‘Connect’ = enable and ‘CHAP Authentication’ = enable, the target CHAP parameters are used
If ‘CHAP Mutual Authentication’ = enable, the Initiator CHAP parameters are used
64 Mellanox Technologies Rev 3.0
10 UEFI PXE Boot
The main differences in UEFI PXE booting is determining the .efi boot file, editing the grub.cfg
and dhcpd.conf
files, and setting the ‘Boot Mode’ to ‘UEFI Mode’ in the
BIOS.
For further information, please refer to the “HowTo Configure PXE Server in UEFI Mode
for IPoIB Network” post in the Mellanox Community ( https://community.mellanox.com
Places Solutions PXE)
Rev 3.0 Mellanox Technologies 65
66
Appendix A: Diskless Machines
Mellanox FlexBoot supports booting diskless machines.
NOTE: The following configuration insturctions apply only to operation systems which do not include an inbox driver, and should be skipped in case an OS with an inbox driver is used.
To enable using an IB/ETH driver, the initrd image must include a device driver module and be configured to load that driver. This can be achieved by adding the device driver module into the initrd image and loading it.
The ‘initrd’ image of some Linux distributions such as SuSE Linux Enterprise Server and
Red Hat Enterprise Linux, cannot be edited prior or during the installation process.
If you need to install Linux distributions over Flexboot, please replace your ‘initrd’ images with the images found at: www.mellanox.com Products InfiniBand/VPI
Drivers FlexBoot (Download Tab).
All OSes which doesn’t include inbox driver should do the following.
OSes which include the inbox driver should skip these configurations
A.1 Case I: InfiniBand Ports
The IB driver requires loading the following modules in the specified order:
ib_addr.ko
ib_core.ko
ib_mad.ko
ib_sa.ko
ib_cm.ko
ib_uverbs.ko
ib_ucm.ko
ib_umad.ko
iw_cm.ko
rdma_cm.ko
rdma_ucm.ko
mlx4_core.ko
mlx4_ib.ko
ib_mthca.ko
ipoib_helper.ko – this module is not required for all OS kernels. Please check the release notes.
ib_ipoib.ko
Mellanox Technologies Rev 3.0
A.1.1 Example: Adding an IB Driver to initrd (Linux)
A.1.1.1 Prerequisites
The FlexBoot image is already programmed on the HCA card
The DHCP server is installed, configured and connected to the client machine
An initrd file.
To add an IB driver into initrd, you need to copy the IB modules to the diskless image.
Your machine needs to be pre-installed with a Mellanox OFED for Linux ISO image
(available for download from www.mellanox.com Products InfiniBand/VPI Drivers
Linux SW/ Drivers) that is appropriate for the kernel version the diskless image will run.
The remainder of this section assumes that Mellanox OFED has been installed on your machine.
A.1.1.2 Adding the IB Driver to the initrd File
NOTE: The following procedure modifies critical files used in the boot procedure. It must be executed by users with expertise in the boot process. Improper application of this pro- cedure may prevent the diskless machine from booting.
1. Back up your current initrd file.
2. Make a new working directory and change to it. host1$ mkdir /tmp/initrd_ib host1$ cd /tmp/initrd_ib
3. Extract the initrd.
host1$ gzip -dc <initrd image> | cpio -id
The initrd files are now located at: /tmp/initrd_ib
4. Create a directory for the InfiniBand modules and copy them. host1$ mkdir -p /tmp/initrd_ib/lib/modules/ib host1$ cd /lib/modules/`uname -r`/updates/kernel/drivers host1$ cp infiniband/core/ib_addr.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_core.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_mad.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_sa.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_cm.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_uverbs.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_ucm.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/ib_umad.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/iw_cm.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/rdma_cm.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/core/rdma_ucm.ko /tmp/initrd_ib/lib/modules/ib host1$ cp net/mlx4/mlx4_core.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/hw/mlx4/mlx4_ib.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/hw/mthca/ib_mthca.ko /tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/ulp/ipoib/ipoib_helper.ko
/tmp/initrd_ib/lib/modules/ib host1$ cp infiniband/ulp/ipoib/ib_ipoib.ko /tmp/initrd_ib/lib/modules/ib
5. Load the IPv6 module. If you do not have it in your initrd , add it using the following command.
Rev 3.0 Mellanox Technologies 67
68 host1$ cp /lib/modules/`uname -r`/kernel/net/ipv6/ipv6.ko
/tmp/initrd_ib/lib/modules
6. Load the modules using the insmod executable. If you do not have it in your initrd , add it using the following command. host1$ cp /sbin/insmod /tmp/initrd_ib/sbin/
7. [Optional] Give your IB device a static IP address. host1$ cp /sbin/ifconfig /tmp/initrd_ib/sbin
8.
[Optional]
Obtain an IP address for the IB device through DHCP. Copy the DHCP client which was compiled specifically to support IB.
To continue with this step, DHCP client needs to be already installed on the machine you are working with.
Copy the DHCP client file and all the relevant files as described below. host1# cp <path to DHCP client>/dhclient /tmp/initrd_ib/sbin host1# cp <path to DHCP client>/dhclient-script /tmp/initrd_ib/sbin host1# mkdir -p /tmp/initrd_ib/var/state/dhcp host1# touch /tmp/initrd_ib/var/state/dhcp/dhclient.leases host1# cp /bin/uname /tmp/initrd_ib/bin host1# cp /usr/bin/expr /tmp/initrd_ib/bin host1# cp /sbin/ifconfig /tmp/initrd_ib/bin host1# cp /bin/hostname /tmp/initrd_ib/bin
9. Create a configuration file for the DHCP client and place it under /tmp/initrd_ib/sbin .
Example of the dclient.conf
# The value indicates a hexadecimal number
# For a ConnectX® device interface "ib0" {send dhcp-client-identifier ff:00:00:00:00:00:02:00:00:02:c9:00:00:02:c9:03:00:00:10:39;
}
10. Add the commands for loading the copied modules into the file init at the point you wish the IB driver to be loaded.
Please pay attention to the following commands order (for loading modules) as it is critical.
Loading “ipoib_helper.ko” is not required for all OS kernels. echo “loading ipv6”
/sbin/insmod /lib/modules/ipv6.ko echo “loading IB driver”
/sbin/insmod /lib/modules/ib/ib_addr.ko
/sbin/insmod /lib/modules/ib/ib_core.ko
/sbin/insmod /lib/modules/ib/ib_mad.ko
/sbin/insmod /lib/modules/ib/ib_sa.ko
/sbin/insmod /lib/modules/ib/ib_cm.ko
/sbin/insmod /lib/modules/ib/ib_uverbs.ko
/sbin/insmod /lib/modules/ib/ib_ucm.ko
/sbin/insmod /lib/modules/ib/ib_umad.ko
/sbin/insmod /lib/modules/ib/iw_cm.ko
/sbin/insmod /lib/modules/ib/rdma_cm.ko
/sbin/insmod /lib/modules/ib/rdma_ucm.ko
/sbin/insmod /lib/modules/ib/mlx4_core.ko
/sbin/insmod /lib/modules/ib/mlx4_ib.ko
/sbin/insmod /lib/modules/ib/ib_mthca.ko
In case of interoperability issues between iSCSI and Large Receive Offload (LRO), set the “ib_ipoib.ko lro” to “0” to disable LRO.
/sbin/insmod /lib/modules/ib/ib_ipoib.ko lro=0
Mellanox Technologies Rev 3.0
11. Assign an IP address to the IB device. Add either a call to ifconfig or to the DHCP client in the init file after loading the modules. If you want to use the DHCP client, then you need to add a call to the DHCP client in the init file after loading the IB modules.
For example:
/sbin/dhclient -cf /sbin/dhclient.conf ib1
12. Save the init file.
13. Close initrd . host1$ cd /tmp/initrd_ib host1$ find ./ | cpio -H newc -o > /tmp/new_initrd_ib.img host1$ gzip /tmp/new_init_ib.img
14. Copy the modified initrd (including the IB driver) which is located at /tmp/new_init_ib.img.gz and rename it properly.
15. Back up your current initrd file.
16. Make a new working directory and change to it.
17. Extract the initrd image.
18. Create a directory for the ConnectX EN modules and copy them.
19. Load the modules using the insmod executable. If you do not have it in your initrd , add it using the following command.
20. Give your Ethernet device a static IP address.
21. Add the commands for loading the copied modules into the file init at the point you wish the Ethernet driver to be loaded.
22. Assign an IP address to the ConnectX EB network interface.
23. Save the init file.
24. Close initrd .
25. Copy the modified initrd (including the Ethernet driver) which is located at /tmp/new_init_en.img.gz and rename it properly.
Rev 3.0 Mellanox Technologies 69
advertisement
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project