advertisement
User’s Guide
OpenStack ® Deployment with SR-IOV Configuration
QLogic ® 3400 and 8400 Series Adapters
83840-546-02 02
User’s Guide—OpenStack ® Deployment with SR-IOV Configuration
QLogic ® 3400 and 8400 Series Adapters
Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only. QLogic Corporation makes no representation nor warranty that such applications are suitable for the specified use without further testing or modification. QLogic Corporation assumes no responsibility for any errors that may appear in this document.
Document Revision History
Rev 01, May 15, 2015
Rev 02, May 22, 2015
Changes Sections Affected
Removed OpenStack release name.
Removed PRELIMINARY from footer.
Added OpenStack release names.
Title page
All
“What is OpenStack?” on page 1
In the last paragraph, clarified the first sentence to indicate that “OpenStack (starting with Juno)…”
In the third bullet, added web site for OpenStack
(Kilo) configuration information.
Added a NOTE about the QLogic OpenStack Utility.
In the second paragraph, removed the reference to
OpenStack (Juno).
“SR-IOV Configuration Using the 3400/8400
ii 83840-546-02 02
1
2
3
4
5
Table of Contents
Preface
Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Downloading Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contact Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Knowledge Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction
Example Architecture
Controller Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Compute Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Network Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Prerequisites
SR-IOV Configuration Using the 3400/8400 Series Adapters
Enabling SR-IOV on the 3400/8400 Series Adapters (Compute Node). . . .
Configuring SR-IOV on the Controller Node . . . . . . . . . . . . . . . . . . . . . . . .
Configuring SR-IOV on the Compute Node . . . . . . . . . . . . . . . . . . . . . . . . .
Configuration Deployment with SR-IOV
Creating an SR-IOV Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Creating the Subnet “subnet2” for the SR-IOV Network . . . . . . . . . . . . . . .
Creating a VM with the SR-IOV Network . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi vii vii
ix ix
iii 83840-546-02 02
6
7
User’s Guide—OpenStack ® Deployment with SR-IOV Configuration
QLogic ® 3400 and 8400 Series Adapters
Testing
Known Issue
iv 83840-546-02 02
Preface
Intended Audience
This guide is intended for OpenStack (June and Kilo) users who want to configure single root input/output virtualization (S R-IOV) with the QLogic 3400/8400 Series
Adapters.
What Is in This Guide
This preface specifies the intended audience, explains the typographic conventions used in this guide, lists related documents, and provides technical support and contact information.
The remainder of this guide is organized into the following chapters:
Chapter 1 Introduction describes the audience for this guide and outlines the
OpenStack architecture.
Chapter 2 Example Architecture
describes the OpenStack 3 node architecture.
Chapter 3 Prerequisites describes what you need to use OpenStack with
SR-IOV configuration using the QLogic 3400/8400 Series Adapters.
Chapter 4 SR-IOV Configuration Using the 3400/8400 Series Adapters
describes how to configure SR-IOV using the QLogic 3400/8400 Series
Adapters.
Chapter 5 Configuration Deployment with SR-IOV
describes how to deploy
OpenStack with SR-IOV.
Chapter 6 Testing provides a sample testing procedure.
Chapter 7 Known Issue describes a known issue with open virtual search
(OVS) and OpenStack.
Related Materials
For information about downloading documentation from the QLogic Web site, see
“Downloading Updates” on page viii .
v 83840-546-02 02
Preface
Documentation Conventions
Documentation Conventions
This guide uses the following documentation conventions:
Text in blue font indicates a hyperlink (jump) to a figure, table, or section in this guide, and links to Web sites are shown in underlined blue . For example:
Table 9-2 lists problems related to the user interface and remote agent.
See “Installation Checklist” on page 6 .
For more information, visit www.qlogic.com
.
Text in bold font indicates user interface elements such as a menu items, buttons, check boxes, or column headings. For example:
Click the Start button, point to Programs, point to Accessories, and then click Command Prompt.
Under Notification Options, select the Warning Alarms check box.
Text in Courier font indicates a file name, directory path, or command line text. For example:
To return to the root directory from anywhere in the file structure:
Type
cd /root and press ENTER.
Enter the following command: sh ./install.bin
Key names and key strokes are indicated with UPPERCASE:
Press CTRL+P.
Press the UP ARROW key.
Text in italics indicates terms, emphasis, variables, or document titles. For example:
For a complete listing of license agreements, refer to the QLogic
Software End User License Agreement.
What are shortcut keys?
To enter the date type mm/dd/yyyy (where mm is the month, dd is the day, and yyyy is the year).
Topic titles between quotation marks identify related topics either within this manual or in the online help, which is also referred to as the help system throughout this document.
vi 83840-546-02 02
Preface
License Agreements
Command syntax conventions include the following:
Plain text indicates items that you must type as shown. For example:
qaucli -pr nic -ei
< > (angle brackets) indicate a variable whose value you must specify.
For example:
<serial_number>
[ ] (square brackets) indicate an optional parameter. For example:
[<file_name>]
means specify a file name, or omit it to select the default file name.
| (vertical bar) indicates mutually exclusive options; select one option only. For example:
on|off
1|2|3|4
... (ellipsis) indicates that the preceding item may be repeated. For example:
x... means one or more instances of x.
[x...] means zero or more instances of x.
( ) (parentheses) and { } (braces) are used to avoid logical ambiguity.
For example:
a|b c
is ambiguous
{(a|b) c}
means a or b, followed by c
{a|(b c)}
means either a, or b c
License Agreements
Refer to the QLogic Software End User License Agreement for a complete listing of all license agreements affecting the QLogic 3400/8400 Series Adapters.
Technical Support
Customers should contact their authorized maintenance provider for technical support of their QLogic products. QLogic-direct customers may contact QLogic
Technical Support; others will be redirected to their authorized maintenance provider. Visit the QLogic support Web site listed in
for the latest firmware and software updates.
For details about available service plans, or for information about renewing and extending your service, visit the Service Program Web page at http://www.qlogic.com/Support/Pages/ServicePrograms.aspx
.
vii 83840-546-02 02
Preface
Technical Support
Downloading Updates
The QLogic Web site provides periodic updates to product firmware, software, and documentation.
To download firmware, software, and documentation:
1.
Go to the QLogic Downloads and Documentation page: http://driverdownloads.qlogic.com
.
2.
Type the QLogic model name in the search box.
3.
In the search results list, locate and select the firmware, software, or documentation for your product.
4.
View the product details Web page to ensure that you have the correct firmware, software, or documentation. For additional information, click
Read Me and Release Notes under Support Files.
5.
Click Download Now.
6.
Save the file to your computer.
7.
If you have downloaded firmware, software, drivers, or boot code, follow the installation instructions in the Readme file.
Instead of typing a model name in the search box, you can perform a guided search as follows:
1.
Click the product type tab: Adapters, Switches, Routers, or ASICs.
2.
Click the corresponding button to search by model or operating system.
3.
Click an item in each selection column to define the search, and then click
Go.
4.
Locate the firmware, software, or document you need, and then click the item’s name or icon to download or open the item.
Training
QLogic Global Training maintains a Web site at www.qlogictraining.com
offering online and instructor-led training for all QLogic products. In addition, sales and technical professionals may obtain Associate and Specialist-level certifications to qualify for additional benefits from QLogic.
viii 83840-546-02 02
Preface
Technical Support
Contact Information
QLogic Technical Support for products under warranty is available during local standard working hours excluding QLogic Observed Holidays. For customers with extended service, consult your plan for available hours. For Support phone numbers, see the Contact Support link at support.qlogic.com
.
Support Headquarters QLogic Corporation
12701 Whitewater Drive
Minnetonka, MN 55343 USA
QLogic Web Site www.qlogic.com
Technical Support Web Site http://support.qlogic.com
Technical Support E-mail [email protected]
Technical Training E-mail [email protected]
Knowledge Database
The QLogic knowledge database is an extensive collection of QLogic product information that you can search for specific solutions. QLogic is constantly adding to the collection of information in the database to provide answers to your most urgent questions. Access the database from the QLogic Support Center: http://support.qlogic.com.
ix 83840-546-02 02
1
Introduction
This chapter describes the audience for this guide and outlines the OpenStack architecture.
Audience
This guide is intended for OpenStack (Juno and Kilo) users who want to configure single root input/output virtualization (SR-IOV) with the QLogic 3400/8400 Series
Adapters.
What is OpenStack?
The OpenStack project is an open source cloud computing platform that supports all types of cloud environments and works as infrastructure as a service (IAAS).
The technology consists of a series of interrelated projects that control pools of processing, storage, and networking resources throughout a data center, which users manage through a web-based dashboard, command line tools, or representational state transfer (REST) APIs.
OpenStack (starting with Juno) adds inbox support to request virtual machine
(VM) access to the virtual network through SR-IOV NIC. With the introduction of
SR-IOV-based NICs, Each SR-IOV port is associated with virtual functions (VFs).
SR-IOV ports are provided by hardware based virtual Ethernet bridging (HW
VEB).
1 83840-546-02 02
2
Example Architecture
This chapter describes the OpenStack 3 node architecture.
OpenStack Nodes
This guide is intended for use with OpenStack 3 node architecture, which consists of the following nodes:
Controller
Compute
Network
These nodes are described in the following sections.
Controller Node
In the example architecture used in this guide, the Controller node runs the
Identity service (Keystone), Image Service (Glance), management portions of the
Compute service (Nova Management) and Networking service (Neutron
Server/modular layer 2 (ML2) plug-in), networking plug-in, and the dashboard
(Horizon). The architecture also includes supporting services such as a database
(mysql), message broker (Rabbitmq), and network time protocol (NTP).
Compute Node
The Compute node runs the hypervisor portion of Compute, which operates tenant virtual machines.
In this architecture, the Compute node uses a kernel-based virtual machine (KVM) as the hypervisor (KVM is the default hypervisor). The Compute node runs the
Networking node plug-in (ML2), layer 2 agent open virtual search (OVS), and NIC switch agent (SR-IOV switch).
Network Node
The Network node runs the Networking plug-in (ML2), layer 3 agent, and DHCP agent. This node also handles external (Internet) connectivity for tenant VMs or the Compute node.
2 83840-546-02 02
3
Prerequisites
This chapter describes what you need to use OpenStack with SR-IOV configuration using the QLogic 3400/8400 Series Adapters.
You need the following knowledge and equipment:
One or more QLogic 3400/8400 Series Adapters
An understanding of OpenStack deployment and experience with
OpenStack 3 node configuration using flat mode networking with virtual LAN
(VLAN) with the ML2/OVS plug-in for public and private (that is, net1) networks.
More information about OpenStack 3 node configuration on Red Hat ® 7 can be found at the following locations:
OpenStack (Juno): http://docs.openstack.org/juno/install-guide/install/yum/content/
OpenStack (Kilo): http://docs.openstack.org/kilo/install-guide/install/yum/content/
The Compute node must have one or more QLogic 3400/8400 Series
Adapters present in system with SR-IOV support enabled.
For the VM instance, QLogic uses a Red Hat 7 KVM Guest image to test
SR-IOV configurations that support the 3400/8400 Series Adapters’ inbox driver, as described here: http://docs.openstack.org/image-guide/content/ch_obtaining_images.html
OpenStack 3 node configuration without SR-IOV support must work with flat networking/VLAN. You must be able to assign floating IP access and log in to the VM.
3 83840-546-02 02
4
SR-IOV Configuration
Using the 3400/8400 Series
Adapters
This chapter describes how to configure SR-IOV using the QLogic 3400/8400
Series Adapters, as described in the following sections:
“Enabling SR-IOV on the 3400/8400 Series Adapters (Compute Node)”
“Configuring SR-IOV on the Controller Node ” on page 5
“Configuring SR-IOV on the Compute Node” on page 7
NOTE
The steps in the following sections can easily be accomplished using the
QLogic provided OpenStack Utility, available for download from: http://www.qlogic.com/solutions/Pages/OpenStack.aspx
. Two packages are provided:
qlgcOpenStack.zip. Standalone package containing source files
qlgcOpenStackEc.zip. Standalone package containing source files and
Eclipse project files to simplify the customization of the QLogic scripts using Eclipse
Enabling SR-IOV on the 3400/8400 Series
Adapters (Compute Node)
To enable SR-IOV on the 3400/8400 Series Adapters in the Compute node:
1.
Ensure that one or more 3400/8400 Series Adapters with SR-IOV support are present on the Compute node.
2.
Install the appropriate bnx2x NIC driver on the Compute node.
3.
Load the NIC drivers by issuing the following command: modprobe bnx2x
4 83840-546-02 02
4–SR-IOV Configuration Using the 3400/8400 Series Adapters
Configuring SR-IOV on the Controller Node
4.
Ensure that the port is up by issuing the following command: ifup ethx / ifconfig ethx up
5.
To create eight VFs, issue the following command: echo 8 > /sys/bus/pci/devices/0000\:0x\:00.x/sriov_numvfs
6.
Verify that the virtual interfaces are visible to host OS by issuing the following command: lspci |grep Qlogic
7.
Enable SR-IOV on the port where the tenant network has been defined.
Configuring SR-IOV on the Controller Node
Since the Neutron server is running on the Controller node, the file configuration for the Neutron server must be changed to enable SR-IOV, as described in the following procedure.
To configure SR-IOV on the Controller node:
1.
Modify the ML2 Neutron plug-in by editing the
/etc/neutron/plugins/ml2/ml2_conf.ini
file on Controller node as follows:
[ml2] tenant_network_types = flat,vlan type_drivers = vlan mechanism_drivers = openvswitch,sriovnicswitch
[ml2_type_vlan] network_vlan_ranges = physnet2:1000:2000,physnet3:2001:2100
[securitygroup] firewall_driver = neutron.agent.firewall.NoopFirewallDriver
5 83840-546-02 02
4–SR-IOV Configuration Using the 3400/8400 Series Adapters
Configuring SR-IOV on the Controller Node
2.
Add the supported PCI vendor VF devices defined by vendor_id:product_id
according to the PCI ID Repository by editing the /etc/neutron/plugins/ml2/ml2_conf_sriov.ini file as follows:
[ml2_sriov] supported_pci_vendor_devs = 14e4:16ad
[sriov_nic] physical_device_mappings = physnet3:p5p2
3.
Change the Neutron server configuration.
The Neutron server must be run with the following two configuration files:
/etc/neutron/plugins/ml2/ml2_conf.in
/etc/neutron/plugins/ml2/ml2_conf_sriov.ini
a.
Navigate to the configure files’ location by issuing the following command:
# cd /usr/lib/systemd/system b.
Edit the neutron-server.service file and add the ml2_conf_sriov.ini
configuration file as follows:
ExecStart=/usr/bin/neutron-server --config-file
/usr/share/neutron/neutron-dist.conf --config-file
/etc/neutron/neutron.conf --config-file
/etc/neutron/plugin.ini --config-file
/etc/neutron/plugins/ml2/ml2_conf_sriov.ini
--log-file
/var/log/neutron/server.log
4.
Restart the Neutron server by issuing the following commands:
# systemctl daemon-reload
# systemctl restart neutron-server.service
6 83840-546-02 02
4–SR-IOV Configuration Using the 3400/8400 Series Adapters
Configuring SR-IOV on the Compute Node
Configuring SR-IOV on the Compute Node
To configure SR-IOV on the Compute note, associate the available VF with the physical network as follows:
1.
Add pci_passthrough_whitelist to the /etc/nova/nova.conf file as follows: pci_passthrough_whitelist={"devname": "p5p2",
"physical_network":"physnet3"}
2.
Restart the OpenStack Nova service by issuing the following command:
# systemctl restart openstack-nova-compute.service
3.
Install the SR-IOV Neutron agent on the Compute node by issuing the following command:
# yum install neutron-sriov-nic-agent
4.
Configure ML2 on the Compute node by editing the node’s ml2_conf.ini file to add the NoopFirewallDriver, as follows:
[securitygroup] firewall_driver = neutron.agent.firewall.NoopFirewallDriver
5.
To run the Neutron SR-IOV NIC agent service, first navigate to the configuration file location by issuing the following command:
# cd /usr/lib/systemd/system
6.
Edit the neutron-sriov-nic-agent.service file and add the ml2_conf_sriov.ini
configuration file as follows:
ExecStart=/usr/bin/neutron-sriov-nic-agent --config-file
/usr/share/neutron/neutron-dist.conf --config-file
/etc/neutron/neutron.conf --config-file
/etc/neutron/plugins/ml2/ml2_conf_sriov.ini
--log-file
/var/log/neutron/sriov-nic-agent.log
7.
Restart the Neutron Open vSwitch agent service by issuing the following command:
# service neutron-plugin-openvswitch-agent restart
7 83840-546-02 02
5
Configuration Deployment with SR-IOV
This chapter describes how to deploy OpenStack with SR-IOV, as follows:
“Creating the Subnet “subnet2” for the SR-IOV Network” on page 9
“Creating the SR-IOV Port” on page 10
“Creating a VM with the SR-IOV Network” on page 11
Creating an SR-IOV Network
To create an SR-IOV network:
1.
Create an SR-IOV network with a VLAN network type by issuing the following command:
# neutron net-create --provider:physical_network=physnet3
--provider:network_type=vlan sriov
2.
Issue the following command to view the configuration for OpenStack 3 nodes:
# [root@network1 ~]# neutron net-list
Following is a sample output.
8 83840-546-02 02
5–Configuration Deployment with SR-IOV
Creating the Subnet “subnet2” for the SR-IOV Network
The values in the name column are described in the following paragraphs.
net1
A private network that provides internal network access for instances of the Compute node.
Public and private networks are configured in the prerequisite section.
sriov
An SR-IOV network for VM access to a virtual network using
SR-IOV NIC.
public
An external network that provides Internet access for instances of the Compute node using a network address translation (NAT)
/floating IP address and a qualified security group.
Creating the Subnet “subnet2” for the SR-IOV
Network
To create the subnet2 subnet for the SR-IOV network:
1.
Create a subnet attached to the SR-IOV network by issuing the following command:
# neutron subnet-create sriov --name subnet2 11.0.0.0/24
2.
Issue the following command to view the subnet network:
# neutron subnet-list
Following is a sample output.
The values in the name column are described in the following paragraphs.
subnet1
A private subnet that uses DHCP
Private subnets are configured in the prerequisite section.
subnet2 public-subnet
An SRI-OV subnet
A subnet for external connectivity
9 83840-546-02 02
5–Configuration Deployment with SR-IOV
Creating the SR-IOV Port
Creating the SR-IOV Port
To create the SR-IOV port:
1.
Issue the following command to create an SR-IOV port:
#neutron port-create <sriov net id> --name <port_name>
--binding:vnic-type direct --device_owner network:dhcp
Following is a sample output.
2.
To show the port information, issue the following command:
# neutron port-show <ID>
Following is a sample output.
Make sure the output shows the correct vif_type and vnic_type.
10 83840-546-02 02
5–Configuration Deployment with SR-IOV
Creating a VM with the SR-IOV Network
Creating a VM with the SR-IOV Network
This section describes how to create a VM with multiple networks: a private network for a floating IP address and an SR-IOV network with direct VM access using a VF.
To create a VM with the SR-IOV network:
1.
Issue the following command to create a VM:
# nova boot --flavor m1.medium --image <image-id> --nic net-id=<net id of private network net1 > --nic port-id=<port id of sriov network from port-create command in
“Configuring SR-IOV on the Compute Node” on page 7
> <VM name>
In this command, a –nic net-id is given for the private network (net1) and a –nic port-id is given for the SR-IOV network with a Red Hat 7
KVM guest image.
Following is a sample output.
2.
Issue the following command to ensure that the VM is up and running:
#nova list
Following is a sample output.
11 83840-546-02 02
5–Configuration Deployment with SR-IOV
Creating a VM with the SR-IOV Network
3.
Assign a floating IP address to the private network to access using ssh.
Following is a sample output.
12 83840-546-02 02
6
Testing
This chapter provides a sample testing procedure.
To verify SR-IOV configuration with OpenStack:
1.
Create two VMs with different ports using the procedures in
and
“Creating a VM with the SR-IOV Network” on page 11 .
This example creates VMs named VNM3 and VNM4.
2.
Log in to the VMs using the floating IP address, which has been assigned to the private network net1 port.
3.
Ideally, the SR-IOV port should get DHCP IP address 11.0.0.x in the VM, but in this case it is not getting the IP address automatically.
Assign an IP address to the SR-IOV port in the VMs using the ifconfig command, as follows:
VNM3—SR-IOV port IP address 11.0.0.8
VNM4—SR-IOV port IP address 11.0.0.9
Following is a sample output for VNM3.
13 83840-546-02 02
6–Testing
Following is a sample output for VNM4.
14 83840-546-02 02
7
Known Issue
This chapter describes a known issue with OVS and OpenStack.
Sometimes OVS version 2.1.2-2 generates a segfault with the OpenStack (Kilo) release.
QLogic has installed latest version OVS-2.3.1 from the following location: http://openvswitch.org/releases/openvswitch-2.3.1.tar.gz
Following is an example of how to compile OVS.
To compile OVS on Red Hat 7.9:
1.
Copy the distribution tar ball (Openvswitch-2.3.1.tar.gz) to the rpm source directory (/root/rpmbuild/SOURCES).
2.
Install the following build prerequisites before compiling Open vSwitch:
gcc
make
python-devel
openssl-devel
kernel-devel
graphviz
kernel-debug-devel
autoconf
automake
rpm-build
redhat-rpm-config
libtool
3.
Extract the spec file from Openvswitch-2.3.1.tar.gz
(Openvswitch-2.3.1/rhel/openvswitch.spec) to the
/root/rpmbuild/SPECS/
folder.
4.
Edit the openvswitch.spec file and remove the Openvswitch-kmod line from the Requires section.
15 83840-546-02 02
7–Known Issue
5.
Issue the following command: rpmbuild –bb /root/rpmbuild/SPECS/openvswitch.spec
This command creates an Open vSwitch 64-bit rpm in the
/root/rpmbuild/RPMS/x86_64
location.
6.
Issue the following command to install rpm: rpm –ivh openvswitch-2.3.1-1.x86_64.rpm
16 83840-546-02 02
Corporate Headquarters QLogic Corporation 26650 Aliso Viejo Parkway Aliso Viejo, CA 92656 949.389.6000 www.qlogic.com
International Offices UK | Ireland | Germany | France | India | Japan | China | Hong Kong | Singapore | Taiwan | Israel
© 2015 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLogic and the QLogic logo are registered trademarks of QLogic Corporation.
OpenStack is a registered trademark of the OpenStack Foundation. Red Hat is a registered trademark of Red Hat, Inc. in the US and other countries. All other brand and product names are trademarks or registered trademarks of their respective owners. Information supplied by QLogic Corporation is believed to be accurate and reliable. QLogic Corporation assumes no responsibility for any errors in this brochure. QLogic Corporation reserves the right, without notice, to make changes in product design or specifications.
advertisement
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project