This guide provides information on installing virtualization software and configuring guest machines on a Red Hat Enterprise Linux virtualization host.

The initial chapters in this guide outline the prerequisites to enable a Red Hat Enterprise Linux host machine to deploy virtualization. System requirements, compatible hardware, support and product restrictions are covered in detail.

Basic host configuration, including mandatory and optional virtualization packages, are covered in

Chapter 5, Installing the virtualization packages .

Guest virtual machine installation is covered in detail starting from Chapter 6, Guest virtual machine installation overview , with procedures for installing fully virtualized Red Hat Enterprise Linux guests

and Windows para-virtualized guests using virt-manager and virsh.

More detailed information on networking, PCI device configuration, SR-IOV, KVM guest timing management, and troubleshooting help for libvirt and SR-IOV is included later in the guide.

Note

This book provides guidance for virtualization host configuration and guest installation. For more detailed system configuration information, refer to the Red Hat Enterprise Linux —

Virtualization Administration Guide .

1.2. Document at ion Suit e

Red Hat offers a wealth of documentation solutions across its various virtualization products.

Coverage of Red Hat Enterprise Linux and its inbuilt virtualization products includes:

Red Hat Enterprise Linux — Virtualization Getting Started Guide : This guide provides an introduction to virtualization concepts, advantages, and tools, and an overview of Red Hat virtualization documentation and products.

Red Hat Enterprise Linux — Virtualization Host Configuration and Guest Installation Guide : This guide covers the installation of virtualization software and configuration of guest machines on a virtualization host.

Red Hat Enterprise Linux — Virtualization Administration Guide : This guide covers administration of hosts, networking, storage, and device and guest management using either virt-manager or virsh as primary configuration tools. This guide also includes a libvirt and QEMU reference, as well as troubleshooting information.

Red Hat Enterprise Linux — Virtualization Security Guide : This guide provides an overview of virtualization security technologies provided by Red Hat. Also included are recommendations for securing hosts, guests, and shared infrastructure and resources in virtualized environments.

Red Hat Enterprise Linux — Virtualization Tuning and Optimization Guide : This guide provides tips, tricks and suggestions for making full use of virtualization performance features and options for your systems and guest virtual machines.

⁠Chapt er 1 . Int roduct ion

Red Hat Enterprise Linux — V2V Guide describes importing virtual machines from KVM, Xen and

VMware ESX/ESX(i) hypervisors to Red Hat Enterprise Virtualization and KVM managed by libvirt.

Note

All of these guides are available at the Red Hat Customer Portal: https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 2. System requirements

This chapter lists system requirements for successfully running virtual machines, referred to as VMs on Red Hat Enterprise Linux 6. Virtualization is available for Red Hat Enterprise Linux 6 on the Intel

64 and AMD64 architecture.

The KVM hypervisor is provided with Red Hat Enterprise Linux 6.

For information on installing the virtualization packages, see Chapter 5, Installing the virtualization packages .

Min imu m syst em req u iremen t s

6 GB free disk space.

2 GB of RAM.

Reco mmen d ed syst em req u iremen t s

One processor core or hyper-thread for the maximum number of virtualized CPUs in a guest virtual machine and one for the host.

2 GB of RAM plus additional RAM for virtual machines.

6 GB disk space for the host, plus the required disk space for each virtual machine.

Most guest operating systems will require at least 6GB of disk space, but the additional storage space required for each guest depends on its image format.

For guest virtual machines using raw images, the guest's total required space (to tal fo r raw

fo rmat) is equal to or greater than the sum of the space required by the guest's raw image files

(i mag es), the 6GB space required by the host operating system (ho st), and the swap space that guest will require (swap).

Eq u at io n 2.1. Calcu lat in g req u ired sp ace f o r g u est virt u al mach in es u sin g raw imag es total for raw format = images + host + swap

For qcow images, you must also calculate the expected maximum storage requirements of the guest (to tal fo r q co w fo rmat), as qcow and qcow2 images grow as required. To allow for this expansion, first multiply the expected maximum storage requirements of the guest (expected

maxi mum g uest sto rag e) by 1.01, and add to this the space required by the host (ho st), and the necessary swap space (swap).

Eq u at io n 2.2. Calcu lat in g req u ired sp ace f o r g u est virt u al mach in es u sin g q co w imag es total for qcow format = (expected maximum guest storage * 1.01) + host + swap

⁠Chapt er 2 . Syst em requirement s

Guest virtual machine requirements are further outlined in the Red Hat Enterprise Linux 6 Virtualization

Administration Guide in Chapter 6. Overcommitting with KVM.

Calcu lat in g swap sp ace

Using swap space can provide additional memory beyond the available physical memory. The swap partition is used for swapping underused memory to the hard drive to speed up memory performance.

The default size of the swap partition is calculated from the physical RAM of the host.

Red Hat Knowledgebase contains an article on safely and efficiently determining an appropriate size for the swap partition, available here: https://access.redhat.com/site/solutions/15244 .

KVM req u iremen t s

The KVM hypervisor requires: an Intel processor with the Intel VT-x and Intel 64 extensions for x86-based systems, or an AMD processor with the AMD-V and the AMD64 extensions.

Refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide to determine if your processor has the virtualization extensions.

St o rag e su p p o rt

The guest virtual machine storage methods are: files on local storage, physical disk partitions, locally connected physical LUNs,

LVM partitions,

NFS shared file systems, iSCSI,

GFS2 clustered file systems,

Fibre Channel-based LUNs, and

Fibre Channel over Ethernet (FCoE).

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 3. KVM guest virtual machine compatibility

To verify whether your processor supports the virtualization extensions and for information on enabling the virtualization extensions if they are disabled, refer to the Red Hat Enterprise Linux

Virtualization Administration Guide.

3.1. Red Hat Ent erprise Linux 6 support limit s

Red Hat Enterprise Linux 6 servers have certain support limits.

The following URLs explain the processor and memory amount limitations for Red Hat Enterprise

Linux:

For host systems: http://www.redhat.com/resourcelibrary/articles/articles-red-hat-enterprise-linux-

6-technology-capabilities-and-limits

For hypervisors: http://www.redhat.com/resourcelibrary/articles/virtualization-limits-rhelhypervisors

Note

Red Hat Enterprise Linux 6.5 now supports 4TiB of memory per KVM guest.

The following URL is a complete reference showing supported operating systems and host and guest combinations: http://www.redhat.com/resourcelibrary/articles/enterprise-linux-virtualization-support

3.2. Support ed CPU Models

Every hypervisor has its own policy for which CPU features the guest will see by default. The set of

CPU features presented to the guest by QEMU/KVM depends on the CPU model chosen in the guest virtual machine configuration. qemu32 and qemu64 are basic CPU models but there are other models (with additional features) available.

Red Hat Enterprise Linux 6 supports the use of the following QEMU CPU model definitions:

<!-- This is only a partial file, only containing the CPU models. The

XML file has more information (including supported features per model) which you can see when you open the file yourself -->

<cpus>

...

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

⁠Chapt er 3. KVM guest virt ual machine compat ibilit y

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</model>

</arch>

</cpus>

⁠Chapt er 3. KVM guest virt ual machine compat ibilit y

Note

A full list of supported CPU models and recognized CPUID flags can also be found using the

q emu-kvm -cpu ? command.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 4. Virtualization restrictions

This chapter covers additional support and product restrictions of the virtualization packages in Red

Hat Enterprise Linux 6.

4 .1. KVM rest rict ions

The following restrictions apply to the KVM hypervisor:

Maximu m vCPUs p er g u est

The maximum amount of virtual CPUs that is supported per guest varies depending on which minor version of Red Hat Enterprise Linux 6 you are using as a host machine. The release of 6.0 introduced a maximum of 64, while 6.3 introduced a maximum of 160.

Currently with the release of 6.7, a maximum of 240 virtual CPUs per guest is supported.

Co n st an t T SC b it

Systems without a Constant Time Stamp Counter require additional configuration. Refer to

Chapter 14, KVM guest timing management for details on determining whether you have a

Constant Time Stamp Counter and configuration steps for fixing any related issues.

Memo ry o verco mmit

KVM supports memory overcommit and can store the memory of guest virtual machines in swap. A virtual machine will run slower if it is swapped frequently. Red Hat Knowledgebase has an article on safely and efficiently determining an appropriate size for the swap partition, available here: https://access.redhat.com/site/solutions/15244 . When KSM is used for memory overcommitting, make sure that the swap size follows the recommendations described in this article.

Important

When device assignment is in use, all virtual machine memory must be statically preallocated to enable DMA with the assigned device. Memory overcommit is therefore not supported with device assignment.

CPU o verco mmit

It is not recommended to have more than 10 virtual CPUs per physical processor core.

Customers are encouraged to use a capacity planning tool in order to determine the CPU overcommit ratio. Estimating an ideal ratio is difficult as it is highly dependent on each workload. For instance, a guest virtual machine may consume 100% CPU on one use case, and multiple guests may be completely idle on another.

Red Hat does not support running more vCPUs to a single guest than the amount of overall physical cores that exist on the system. While Hyperthreads can be considered as cores, their performance can also vary from one scenario to the next, and they should not be expected to perform as well as regular cores.

Refer to the Red Hat Enterprise Linux Virtualization Administration Guide for tips and recommendations on overcommitting CPUs.

Virt u aliz ed SCSI d evices

⁠Chapt er 4 . Virt ualizat ion rest rict ions

SCSI emulation is not supported with KVM in Red Hat Enterprise Linux.

Virt u aliz ed IDE d evices

KVM is limited to a maximum of four virtualized (emulated) IDE devices per guest virtual machine.

PCI d evices

Red Hat Enterprise Linux 6 supports 32 PCI device slots per virtual machine, and 8 PCI functions per device slot. This gives a theoretical maximum of 256 PCI functions per guest when multi-function capabilities are enabled.

However, this theoretical maximum is subject to the following limitations:

Each virtual machine supports a maximum of 8 assigned device functions.

4 PCI device slots are configured with 5 emulated devices (two devices are in slot 1) by default. However, users can explicitly remove 2 of the emulated devices that are configured by default if the guest operating system does not require them for operation

(the video adapter device in slot 2; and the memory balloon driver device in the lowest available slot, usually slot 3). This gives users a supported functional maximum of 30

PCI device slots per virtual machine.

The following restrictions also apply to PCI device assignment:

PCI device assignment (attaching PCI devices to virtual machines) requires host systems to have AMD IOMMU or Intel VT-d support to enable device assignment of PCI-e devices.

For parallel/legacy PCI, only single devices behind a PCI bridge are supported.

Multiple PCIe endpoints connected through a non-root PCIe switch require ACS support in the PCIe bridges of the PCIe switch. To disable this restriction, edit the

/etc/l i bvi rt/q emu. co nf file and insert the line: relaxed_acs_check=1

Red Hat Enterprise Linux 6 has limited PCI configuration space access by guest device drivers. This limitation could cause drivers that are dependent on PCI configuration space to fail configuration.

Red Hat Enterprise Linux 6.2 introduced interrupt remapping as a requirement for PCI device assignment. If your platform does not provide support for interrupt remapping, circumvent the KVM check for this support with the following command as the root user at the command line prompt:

# echo 1 >

/sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

Mig rat io n rest rict io n s

Device assignment refers to physical devices that have been exposed to a virtual machine, for the exclusive use of that virtual machine. Because device assignment uses hardware on the specific host where the virtual machine runs, migration and save/restore are not supported when device assignment is in use. If the guest operating system supports hotplugging, assigned devices can be removed prior to the migration or save/restore operation to enable this feature.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Live migration is only possible between hosts with the same CPU type (that is, Intel to Intel or AMD to AMD only).

For live migration, both hosts must have the same value set for the No eXecution (NX) bit, either o n or o ff.

For migration to work, cache= no ne must be specified for all block devices opened in write mode.

Warning

Failing to include the cache= no ne option can result in disk corruption.

St o rag e rest rict io n s

There are risks associated with giving guest virtual machines write access to entire disks or block devices (such as /d ev/sd b). If a guest virtual machine has access to an entire block device, it can share any volume label or partition table with the host machine. If bugs exist in the host system's partition recognition code, this can create a security risk. Avoid this risk by configuring the host machine to ignore devices assigned to a guest virtual machine.

Warning

Failing to adhere to storage restrictions can result in risks to security.

SR- IO V rest rict io n s

SR-IOV is only thoroughly tested with the following devices (other SR-IOV devices may work but have not been tested at the time of release):

Intel® 82576NS Gigabit Ethernet Controller (i g b driver)

Intel® 82576EB Gigabit Ethernet Controller (i g b driver)

Intel® 82599ES 10 Gigabit Ethernet Controller (i xg be driver)

Intel® 82599EB 10 Gigabit Ethernet Controller (i xg be driver)

Co re d u mp in g rest rict io n s

Because core dumping is currently implemented on top of migration, it is not supported when device assignment is in use.

4 .2. Applicat ion rest rict ions

There are aspects of virtualization which make it unsuitable for certain types of applications.

Applications with high I/O throughput requirements should use the para-virtualized drivers for fullyvirtualized guests. Without the para-virtualized drivers certain applications may be unpredictable under heavy I/O loads.

The following applications should be avoided due to high I/O requirements:

kd u mp server

⁠Chapt er 4 . Virt ualizat ion rest rict ions

n et d u mp server

You should carefully evaluate applications and tools that heavily utilize I/O or those that require realtime performance. Consider the para-virtualized drivers or PCI device assignment for increased I/O

performance. Refer to Chapter 10, KVM Para-virtualized (virtio) Drivers for more information on the para-

virtualized drivers for fully virtualized guests. Refer to Chapter 12, PCI device assignment for more

information on PCI device assignment.

Applications suffer a small performance loss from running in virtualized environments. The performance benefits of virtualization through consolidating to newer and faster hardware should be evaluated against the potential application performance issues associated with using virtualization.

4 .3. Ot her rest rict ions

For the list of all other restrictions and issues affecting virtualization read the Red Hat Enterprise Linux

6 Release Notes. The Red Hat Enterprise Linux 6 Release Notes cover the present new features, known issues and restrictions as they are updated or discovered.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 5. Installing the virtualization packages

Before you can use virtualization, the virtualization packages must be installed on your computer.

Virtualization packages can be installed either during the host installation sequence or after host installation using Subscription Manager.

The KVM hypervisor uses the default Red Hat Enterprise Linux kernel with the kvm kernel module.

5.1. Configuring a Virt ualizat ion Host inst allat ion

This section covers installing virtualization tools and virtualization packages as part of a fresh Red

Hat Enterprise Linux installation.

Note

The Red Hat Enterprise Linux Installation Guide, available from https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/ , covers installing

Red Hat Enterprise Linux in detail.

Pro ced u re 5.1. In st allin g t h e virt u aliz at io n p ackag e g ro u p

1. Lau n ch t h e Red Hat En t erp rise Lin u x 6 in st allat io n p ro g ram

Start an interactive Red Hat Enterprise Linux 6 installation from the Red Hat Enterprise Linux

Installation CD-ROM, DVD or PXE.

2. Co n t in u e in st allat io n u p t o p ackag e select io n

Complete the other steps up to the package selection step.

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

Fig u re 5.1. T h e Red Hat En t erp rise Lin u x p ackag e select io n screen

Select the Vi rtual i zati o n Ho st server role to install a platform for guest virtual machines. Alternatively, ensure that the C usto mi ze No w radio button is selected before proceeding, to specify individual packages.

3. Select t h e Vi rtual i zati o n p ackag e g ro u p .

This selects the qemu-kvm emulator, vi rt-manag er, l i bvi rt and vi rt-vi ewer for installation.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 5.2. T h e Red Hat En t erp rise Lin u x p ackag e select io n screen

Note

If you wish to create virtual machines in a graphical user interface (virt - man ag er) later, you should also select the G eneral P urpo se D eskto p package group.

Cu st o miz e t h e p ackag es ( if req u ired )

Customize the Vi rtual i zati o n group if you require other virtualization packages.

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

Fig u re 5.3. T h e Red Hat En t erp rise Lin u x p ackag e select io n screen

Click on the C l o se button, then the Next button to continue the installation.

When the installation is complete, reboot the system.

Important

You require a valid virtualization entitlement to receive updates for the virtualization packages.

In st allin g KVM p ackag es wit h Kickst art f iles

Kickstart files allow for large, automated installations without a user manually installing each individual host system. This section describes how to create and use a Kickstart file to install Red Hat

Enterprise Linux with the Virtualization packages.

In the %packag es section of your Kickstart file, append the following package groups:

@ virtualization

@ virtualization-client

@ virtualization-platform

@ virtualization-tools

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

For more information about Kickstart files, refer to the Red Hat Enterprise Linux Installation Guide, available from https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/ .

5.2. Inst alling virt ualizat ion packages on an exist ing Red Hat

Ent erprise Linux syst em

This section describes the steps for installing the KVM hypervisor on a working Red Hat Enterprise

Linux 6 or newer system.

To install the packages, your machines must be registered. To register via Red Hat Subscription

Manager, run the subscri pti o n-manag er reg i ster command and follow the prompts.

If you do not have a valid Red Hat subscription, visit the Red Hat online store to obtain one.

Note

Red Hat Network (RHN) has now been deprecated. Subscription Manager should now be used for registration tasks.

In st allin g t h e virt u aliz at io n p ackag es wit h yum

To use virtualization on Red Hat Enterprise Linux you require at least the q emu-kvm and q emu-i mg packages. These packages provide the user-level KVM emulator and disk image manager on the host Red Hat Enterprise Linux system.

To install the q emu-kvm and q emu-i mg packages, run the following command:

# yum install qemu-kvm qemu-img

Several additional virtualization management packages are also available:

Reco mmen d ed virt u aliz at io n p ackag es python-virtinst

Provides the vi rt-i nstal l command for creating virtual machines.

libvirt

The libvirt package provides the server and host side libraries for interacting with hypervisors and host systems. The libvirt package provides the l i bvi rtd daemon that handles the library calls, manages virtual machines and controls the hypervisor.

libvirt-python

The libvirt-python package contains a module that permits applications written in the Python programming language to use the interface supplied by the libvirt API.

virt-manager

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

vi rt-manag er, also known as Virt u al Mach in e Man ag er, provides a graphical tool for administering virtual machines. It uses libvirt-client library as the management API.

libvirt-client

The libvirt-client package provides the client-side APIs and libraries for accessing libvirt servers. The libvirt-client package includes the vi rsh command line tool to manage and control virtual machines and hypervisors from the command line or a special virtualization shell.

Install all of these recommended virtualization packages with the following command:

# yum install virt-manager libvirt libvirt-python python-virtinst libvirt-client

In st allin g Virt u aliz at io n p ackag e g ro u p s

The virtualization packages can also be installed from package groups. The following table describes the virtualization package groups and what they provide.

Note

Note that the q emu-i mg package is installed as a dependency of the Vi rtual i zati o n package group if it is not already installed on the system. It can also be installed manually with the yum i nstal l q emu-i mg command as described previously.

T ab le 5.1. Virt u aliz at io n Packag e G ro u p s

Packag e G ro u p Descrip t io n Man d at o ry

Packag es qemu-kvm

O p t io n al Packag es

Virtualization

Virtualization Client

Virtualization Platform

Virtualization Tools

Provides an environment for hosting virtual machines

Clients for installing and managing virtualization instances

Provides an interface for accessing and controlling virtual machines and containers

Tools for offline virtual image management python-virtinst, virtmanager, virt-viewer qemu-guest-agent, qemu-kvm-tools virt-top libvirt, libvirt-client, virtwho, virt-what libguestfs fence-virtd-libvirt, fence-virtd-multicast, fence-virtd-serial, libvirt-cim, libvirt-java, libvirt-qmf, libvirt-snmp, perl-Sys-Virt libguestfs-java, libguestfs-tools, virtv2v

To install a package group, run the yum g ro upi nstal l <g ro upname> command. For instance, to install the Virt u aliz at io n T o o ls package group, run the yum g ro upi nstal l

"Vi rtual i zati o n T o o l s" command.

5.3. Regist ering t he hypervisor and virt ual machine

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

5.3. Regist ering t he hypervisor and virt ual machine

Red Hat Enterprise Linux 6 and 7 require that every guest virtual machine is mapped to a specific hypervisor in order to ensure that every guest is allocated the same level of subscription service. To do this you need to install a subscription agent that automatically detects all guest Virtual Machines

(VMs) on each KVM hypervisor that is installed and registered, which in turn will create a mapping file that sits on the host. This mapping file ensures that all guest VMs receive the following benefits:

Subscriptions specific to virtual systems are readily available and can be applied to all of the associated guest VMs.

All subscription benefits that can be inherited from the hypervisor are readily available and can be applied to all of the associated guest VMs.

Note

The information provided in this chapter is specific to Red Hat Enterprise Linux subscriptions only. If you also have a Red Hat Enterprise Virtualization subscription, or a Red Hat Satellite subscription, you should also consult the virt-who information provided with those subscriptions. More information on Red Hat Subscription Management can also be found in the Red Hat Subscription Management Guide found on the customer portal.

5.3.1. Inst alling virt -who on t he host physical machine

1. Reg ist er t h e KVM h yp erviso r

[o pti o ns] command in a terminal as the root user on the host physical machine. More options are available using the # subscri pti o n-manag er reg i ster --hel p menu. In cases where you are using a username and password, use the credentials that are known to the subscription manager. If this is your very first time subscribing and you do not have a user account, contact customer support. For example to register the VM as 'admin' with

'secret' as a password, you would send the following command:

[root@ rhel-server ~]# subscri pti o n-manag er reg i ster -- username= admin --passwo rd = secret --auto -attach --type= hypervi so r

2. In st all t h e virt - wh o p ackag es

Install the virt-who packages, by running the following command in a terminal as root on the host physical machine:

[root@ rhel-server ~]# yum i nstal l vi rt-who

3. Creat e a virt - wh o co n f ig u rat io n f ile

Add a configuration file in the /etc/vi rt-who . d / directory. It does not matter what the name of the file is, but you should give it a name that makes sense and the file must be located in the /etc/vi rt-who . d / directory. Inside that file add the following snippet and remember to save the file before closing it.

[libvirt] type=libvirt

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

4. St art t h e virt - wh o service

Start the virt-who service by running the following command in a terminal as root on the host physical machine:

[root@ virt-who ~]# systemctl start virt-who.service

[root@ virt-who ~]# systemctl enable virt-who.service

5. Co n f irm virt - wh o service is receivin g g u est in f o rmat io n

At this point, the virt-who service will start collecting a list of domains from the host. Check the

/var/l o g /rhsm/rhsm. l o g file on the host physical machine to confirm that the the file contains a list of the guest VMs. For example:

2015-05-28 12:33:31,424 DEBUG: Libvirt domains found: [{'guestId':

'58d59128-cfbb-4f2c-93de-230307db2ce0', 'attributes': {'active': 0,

'virtWhoType': 'libvirt', 'hypervisorType': 'QEMU'}, 'state': 5}]

Pro ced u re 5.2. Man ag in g t h e su b scrip t io n o n t h e cu st o mer p o rt al

1. Su b scrib in g t h e h yp erviso r

As the virtual machines will be receiving the same subscription benefits as the hypervisor, it is important that the hypervisor has a valid subscription and that the subscription is available for the VMs to use.

a. Lo g in t o t h e cu st o mer p o rt al

Subscri pti o ns button at the top of the page.

b. Click t h e Syst ems lin k

In the Subscri ber Invento ry section (towards the bottom of the page), click

Systems link.

c. Select t h e h yp erviso r

On the Systems page, there is a table of all subscribed systems. Click on the name of the hypervisor (localhost.localdomain for example). In the details page that opens, click Attach a subscri pti o n and select all the subscriptions listed. Click Attach

Sel ected . This will attach the host's physical subscription to the hypervisor so that the guests can benefit from the subscription.

2. Su b scrib in g t h e g u est virt u al mach in es - f irst t ime u se

This step is for those who have a new subscription and have never subscribed a guest virtual machine before. If you are adding virtual machines, skip this step. To consume the subscription assigned to the hypervisor profile on the machine running the virt-who service, auto subscribe by running the following command in a terminal, on the guest virtual machine as root.

[root@ virt-who ~]# subscription-manager attach --auto

3. Su b scrib in g ad d it io n al g u est virt u al mach in es

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

If you just subscribed a for the first time, skip this step. If you are adding additional virtual machines, it should be noted that running this command will not necessarily re-attach the same subscriptions to the guest virtual machine. This is because removing all subscriptions then allowing auto attach to resolve what is necessary for a given guest virtual machine may result in different subscriptions consumed than before. This may not have any effect on your system, but it is something you should be aware about. If you used a manual attachment procedure to attach the virtual machine, which is not described below, you will need to reattach those virtual machines manually as the auto-attach will not work. Use the following command as root in a terminal to first remove the subscriptions for the old guests and then use the auto-attach to attach subscriptions to all the guests. Run these commands on the guest virtual machine.

[root@ virt-who ~]# subscription-manager remove --all

[root@ virt-who ~]# subscription-manager attach --auto

4. Co n f irm su b scrip t io n s are at t ach ed

Confirm that the subscription is attached to the hypervisor by running the following command as root in a terminal on the guest virtual machine:

[root@ virt-who ~]# subscription-manager list --consumed

Output similar to the following will be displayed. Pay attention to the Subscription Details. It should say 'Subscription is current'.

[root@ virt-who ~]# subscri pti o n-manag er l i st --co nsumed

+-------------------------------------------+

Consumed Subscriptions

+-------------------------------------------+

Subscription Name: Awesome OS with unlimited virtual guests

Provides: Awesome OS Server Bits

SKU: awesomeos-virt-unlimited

Contract: 0

Account: ######### Your account number #####

Serial: ######### Your serial number ######

Pool ID: XYZ123

Provides Management: No

Active: True

Quantity Used: 1

Service Level:

Service Type:

Status Details: Subscription is current

Subscription Type:

Starts: 01/01/2015

Ends: 12/31/2015

System Type: Virtual

The ID for the subscription to attach to the system is displayed here. You will need this

ID if you need to attach the subscription manually.

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

Indicates if your subscription is current. If your subscription is not current, an error message appears. One example is Guest has not been reported on any host and is using a temporary unmapped guest subscription. In this case the guest needs to be

subscribed. In other cases, use the information as indicated in Section 5.3.5.2, “ I have subscription status errors, what do I do?” .

5. Reg ist er ad d it io n al g u est s

When you install new guest VMs on the hypervisor, you must register the new VM and use the subscription attached to the hypervisor, by running the following commands in a terminal as root on the guest virtual machine:

[root@ server1 ~]# subscription-manager register

[root@ server1 ~]# subscription-manager attach --auto

[root@ server1 ~]# subscription-manager list --consumed

5.3.2. Regist ering a new guest virt ual machine

In cases where a new guest virtual machine is to be created on a host that is already registered and running, the virt-who service must also be running. This ensures that the virt-who service maps the guest to a hypervisor, so the system is properly registered as a virtual system. To register the virtual machine, run the following command as root in a terminal:

[root@ virt-server ~]# subscri pti o n-manag er reg i ster --username= ad mi n -- passwo rd = secret --auto -attach

5.3.3. Removing a guest virt ual machine ent ry

If the guest virtual machine is running, unregister the system, by running the following command in a terminal window as root on the guest:

[root@ virt-guest ~]# subscri pti o n-manag er unreg i ster

If the system has been deleted, however, the virtual service cannot tell whether the service is deleted or paused. In that case, you must manually remove the system from the server side, using the following steps:

1. Lo g in t o t h e Su b scrip t io n Man ag er

The Subscription Manager is located on the Red Hat Customer Portal . Login to the Customer

Portal using your username and password, by clicking the login icon at the top of the screen.

2. Click t h e Su b scrip t io n s t ab

Click the Subscri pti o ns tab.

3. Click t h e Syst ems lin k

Scroll down the page and click the Systems link.

4. Delet e t h e syst em

To delete the system profile, locate the specified system's profile in the table, select the check box beside its name and click D el ete.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

5.3.4 . Inst alling virt -who manually

This section will describe how to manually attach the subscription provided by the hypervisor.

Pro ced u re 5.3. Ho w t o at t ach a su b scrip t io n man u ally

1. List su b scrip t io n in f o rmat io n an d f in d t h e Po o l ID

First you need to list the available subscriptions which are of the virtual type. Run the following command in a terminal as root:

[root@ server1 ~]# subscri pti o n-manag er l i st --avai l --match- i nstal l ed | g rep ' Vi rtual ' -B12

Subscription Name: Red Hat Enterprise Linux ES (Basic for

Virtualization)

Provides: Red Hat Beta

Oracle Java (for RHEL Server)

Red Hat Enterprise Linux Server

SKU: -------

Pool ID: XYZ123

Available: 40

Suggested: 1

Service Level: Basic

Service Type: L1-L3

Multi-Entitlement: No

Ends: 01/02/2017

System Type: Virtual

Note the Pool ID displayed. Copy this ID as you will need it in the next step.

2. At t ach t h e su b scrip t io n wit h t h e Po o l ID

Using the Pool ID you copied in the previous step run the attach command. Replace the Pool

ID XYZ 123 with the Pool ID you retrieved. Run the following command in a terminal as root:

[root@ server1 ~]# subscri pti o n-manag er attach --po o l = XY Z123

Successfully attached a subscription for: Red Hat Enterprise Linux

ES (Basic for Virtualization)

5.3.5. T roubleshoot ing virt -who

5 .3.5 .1 . Why is t he hype rviso r st at us re d?

Scenario: On the server side, you deploy a guest on a hypervisor that does not have a subscription.

24 hours later, the hypervisor displays its status as red. To remedy this situation you must get a subscription for that hypervisor. Or, permanently migrate the guest to a hypervisor with a subscription.

5 .3.5 .2 . I have subscript io n st at us e rro rs, what do I do ?

Scenario: Any of the following error messages display:

System not properly subscribed

⁠Chapt er 5. Inst alling t he virt ualizat ion packages

Status unknown

Late binding of a guest to a hypervisor through virt-who (host/guest mapping)

To find the reason for the error open the virt-who log file, named rhsm. l o g , located in the

/var/l o g /rhsm/ directory.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 6. Guest virtual machine installation overview

After you have installed the virtualization packages on the host system you can create guest operating systems. This chapter describes the general processes for installing guest operating systems on virtual machines. You can create guest virtual machines using the New button in virt -

man ag er or use the command line interface vi rt-i nstal l . Both methods are covered by this chapter.

Detailed installation instructions are available in the following chapters for specific versions of Red

Hat Enterprise Linux and Microsoft Windows.

6.1. Guest virt ual machine prerequisit es and considerat ions

Various factors should be considered before creating any guest virtual machines. Not only should the role of a virtual machine be considered before deployment, but regular ongoing monitoring and assessment based on variable factors (load, amount of clients) should be performed. Some factors include:

Perf o rman ce

Guest virtual machines should be deployed and configured based on their intended tasks.

Some guest systems (for instance, guests running a database server) may require special performance considerations. Guests may require more assigned CPUs or memory based on their role and projected system load.

In p u t /O u t p u t req u iremen t s an d t yp es o f In p u t /O u t p u t

Some guest virtual machines may have a particularly high I/O requirement or may require further considerations or projections based on the type of I/O (for instance, typical disk block size access, or the amount of clients).

St o rag e

Some guest virtual machines may require higher priority access to storage or faster disk types, or may require exclusive access to areas of storage. The amount of storage used by guests should also be regularly monitored and taken into account when deploying and maintaining storage.

Net wo rkin g an d n et wo rk in f rast ru ct u re

Depending upon your environment, some guest virtual machines could require faster network links than other guests. Bandwidth or latency are often factors when deploying and maintaining guests, especially as requirements or load changes.

Req u est req u iremen t s

SCSI requests can only be issued to guest virtual machines on virtio drives if the virtio drives are backed by whole disks, and the disk device parameter is set to l un, as shown in the following example:

<emulator>/usr/libexec/qemu-kvm</emulator>

6.2. Creat ing guest s wit h virt -inst all

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

You can use the vi rt-i nstal l command to create guest virtual machines from the command line.

vi rt-i nstal l is used either interactively or as part of a script to automate the creation of virtual machines. Using vi rt-i nstal l with Kickstart files allows for unattended installation of virtual machines.

The vi rt-i nstal l tool provides a number of options that can be passed on the command line. To see a complete list of options run the following command:

# virt-install --help

Note that you need root privileges in order for vi rt-i nstal l commands to complete successfully.

The vi rt-i nstal l man page also documents each command option and important variables.

q emu-i mg is a related command which may be used before vi rt-i nstal l to configure storage options.

An important option is the --graphics option which allows graphical installation of a virtual machine.

Examp le 6 .1. Usin g virt - in st all t o in st all a Red Hat En t erp rise Lin u x 5 g u est virt u al mach in e

This example creates a Red Hat Enterprise Linux 5 guest: virt-install \

--name=guest1-rhel5-64 \

--file=/var/lib/libvirt/images/guest1-rhel5-64.dsk \

--file-size=8 \

--nonsparse --graphics spice \

--vcpus=2 --ram=2048 \

--location=http://example1.com/installation_tree/RHEL5.6-Serverx86_64/os \

--network bridge=br0 \

--os-type=linux \

--os-variant=rhel5.4

Ensure that you select the correct os-type for your operating system when running this command.

Refer to man vi rt-i nstal l for more examples.

Note

When installing a Windows guest with vi rt-i nstal l , the --o s-type= windows option is recommended. This option prevents the CD-ROM from disconnecting when rebooting during the installation procedure. The --o s-vari ant option further optimizes the configuration for a specific guest operating system.

6.3. Creat ing guest s wit h virt -manager

vi rt-manag er, also known as Virtual Machine Manager, is a graphical tool for creating and managing guest virtual machines.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Pro ced u re 6 .1. Creat in g a g u est virt u al mach in e wit h virt - man ag er

1. O p en virt - man ag er

Start vi rt-manag er. Launch the Virt u al Mach in e Man ag er application from the

Ap p licat io n s menu and Syst em T o o ls submenu. Alternatively, run the vi rt-manag er command as root.

2. O p t io n al: O p en a remo t e h yp erviso r

Select the hypervisor and click the C o nnect button to connect to the remote hypervisor.

3. Creat e a n ew virt u al mach in e

The virt - man ag er window allows you to create a new virtual machine. Click the C reate a

new vi rtual machi ne button (

Figure 6.1, “ Virtual Machine Manager window”

) to open the

New VM wizard.

Fig u re 6 .1. Virt u al Mach in e Man ag er win d o w

The New VM wizard breaks down the virtual machine creation process into five steps: a. Naming the guest virtual machine and choosing the installation type b. Locating and configuring the installation media c. Configuring memory and CPU options d. Configuring the virtual machine's storage e. Configuring networking, architecture, and other hardware settings

Ensure that vi rt-manag er can access the installation media (whether locally or over the network) before you continue.

4. Sp ecif y n ame an d in st allat io n t yp e

The guest virtual machine creation process starts with the selection of a name and installation type. Virtual machine names can have underscores (_), periods (. ), and hyphens

(-).

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

Fig u re 6 .2. Name virt u al mach in e an d select in st allat io n met h o d

Type in a virtual machine name and choose an installation type:

Lo cal in st all med ia ( ISO imag e o r CDRO M)

This method uses a CD-ROM, DVD, or image of an installation disk (for example,

. i so ).

Net wo rk In st all ( HT T P, FT P, o r NFS)

This method involves the use of a mirrored Red Hat Enterprise Linux or Fedora installation tree to install a guest. The installation tree must be accessible through either HTTP, FTP, or NFS.

Net wo rk Bo o t ( PXE)

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

This method uses a Preboot eXecution Environment (PXE) server to install the guest virtual machine. Setting up a PXE server is covered in the Deployment Guide. To install via network boot, the guest must have a routable IP address or shared network device. For information on the required networking configuration for PXE

installation, refer to Section 6.4, “ Installing guest virtual machines with PXE” .

Imp o rt exist in g d isk imag e

This method allows you to create a new guest virtual machine and import a disk image (containing a pre-installed, bootable operating system) to it.

Click Fo rward to continue.

5. Co n f ig u re in st allat io n

Next, configure the O S t yp e and Versio n of the installation. Ensure that you select the appropriate OS type for your virtual machine. Depending on the method of installation, provide the install URL or existing storage path.

Fig u re 6 .3. Remo t e in st allat io n URL

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

Fig u re 6 .4 . Lo cal ISO imag e in st allat io n

6. Co n f ig u re CPU an d memo ry

The next step involves configuring the number of CPUs and amount of memory to allocate to the virtual machine. The wizard shows the number of CPUs and amount of memory you can allocate; configure these settings and click Fo rward .

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 6 .5. Co n f ig u rin g CPU an d Memo ry

7. Co n f ig u re st o rag e

Assign storage to the guest virtual machine.

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

Fig u re 6 .6 . Co n f ig u rin g virt u al st o rag e

If you chose to import an existing disk image during the first step, vi rt-manag er will skip this step.

Assign sufficient space for your virtual machine and any applications it requires, then click

Fo rward to continue.

8. Fin al co n f ig u rat io n

Verify the settings of the virtual machine and click Fi ni sh when you are satisfied; doing so will create the virtual machine with default networking settings, virtualization type, and architecture.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 6 .7. Verif yin g t h e co n f ig u rat io n

If you prefer to further configure the virtual machine's hardware first, check the Cu st o miz e

co n f ig u rat io n b ef o re in st all box first before clicking Fi ni sh. Doing so will open another wizard that will allow you to add, remove, and configure the virtual machine's hardware settings.

After configuring the virtual machine's hardware, click Appl y. vi rt-manag er will then create the virtual machine with your specified hardware settings.

6.4 . Inst alling guest virt ual machines wit h PXE

Req u iremen t s

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

PXE guest installation requires a PXE server running on the same subnet as the guest virtual machines you wish to install. The method of accomplishing this depends on how the virtual machines are connected to the network. Contact Support if you require assistance setting up a PXE server.

PXE in st allat io n wit h virt - in st all

vi rt-i nstal l PXE installations require both the --netwo rk= bri d g e: i nstal l ati o n parameter, where installation is the name of your bridge, and the --pxe parameter.

By default, if no network is found, the guest virtual machine attempts to boot from alternative bootable devices. If there is no other bootable device found, the guest pauses. You can use the q emu - kvm boot parameter reboot-timeout to allow the guest to retry booting if no bootable device is found, like so:

# qemu-kvm -boot reboot-timeout=1000

Examp le 6 .2. Fu lly- virt u aliz ed PXE in st allat io n wit h virt - in st all

# virt-install --hvm --connect qemu:///system \

--network=bridge:installation --pxe --graphics spice \

--name rhel6-machine --ram=756 --vcpus=4 \

--os-type=linux --os-variant=rhel6 \

--disk path=/var/lib/libvirt/images/rhel6-machine.img,size=10

Note that the command above cannot be executed in a text-only environment. A fully-virtualized (-

-hvm) guest can only be installed in a text-only environment if the --location and --extra-

args "console=console_type" are provided instead of the --graphics spice parameter.

Pro ced u re 6 .2. PXE in st allat io n wit h virt - man ag er

Select PXE

Select PXE as the installation method and follow the rest of the steps to configure the OS type, memory, CPU and storage settings.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 6 .8. Select in g t h e in st allat io n met h o d

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

Fig u re 6 .9 . Select in g t h e in st allat io n t yp e

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

4 0

Fig u re 6 .10. Sp ecif yin g virt u aliz ed h ard ware d et ails

⁠Chapt er 6 . G uest virt ual machine inst allat ion overview

Fig u re 6 .11. Sp ecif yin g st o rag e d et ails

St art t h e in st allat io n

The installation is ready to start.

4 1

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 6 .12. Fin aliz in g virt u al mach in e d et ails

A DHCP request is sent and if a valid PXE server is found the guest virtual machine's installation processes will start.

4 2

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Chapter 7. Installing a Red Hat Enterprise Linux 6 guest virtual machine on a Red Hat Enterprise Linux 6 host

This chapter covers how to install a Red Hat Enterprise Linux 6 guest virtual machine on a Red Hat

Enterprise Linux 6 host.

These procedures assume that the KVM hypervisor and all other required packages are installed and the host is configured for virtualization.

Note

For more information on installing the virtualization packages, refer to Chapter 5, Installing the virtualization packages .

7.1. Creat ing a Red Hat Ent erprise Linux 6 guest wit h local inst allat ion media

This procedure covers creating a Red Hat Enterprise Linux 6 guest virtual machine with a locally stored installation DVD or DVD image. DVD images are available from http://access.redhat.com

for

Red Hat Enterprise Linux 6.

Pro ced u re 7.1. Creat in g a Red Hat En t erp rise Lin u x 6 g u est virt u al mach in e wit h virt man ag er

1. O p t io n al: Prep arat io n

Prepare the storage environment for the virtual machine. For more information on preparing storage, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.

Important

Various storage types may be used for storing guest virtual machines. However, for a virtual machine to be able to use migration features the virtual machine must be created on networked storage.

Red Hat Enterprise Linux 6 requires at least 1GB of storage space. However, Red Hat recommends at least 5GB of storage space for a Red Hat Enterprise Linux 6 installation and for the procedures in this guide.

2. O p en virt - man ag er an d st art t h e wiz ard

Open virt-manager by executing the vi rt-manag er command as root or opening

Appl i cati o ns → System T o o l s → Vi rtual Machi ne Manag er.

4 3

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 7.1. T h e Virt u al Mach in e Man ag er win d o w

Click on the C reate a new vi rtual machi ne button to start the new virtualized guest wizard.

4 4

Fig u re 7.2. T h e Creat e a n ew virt u al mach in e b u t t o n

The New VM window opens.

3. Name t h e virt u al mach in e

Virtual machine names can contain letters, numbers and the following characters: '_', '. ' and

'-'. Virtual machine names must be unique for migration and cannot consist only of numbers.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Choose the Lo cal i nstal l med i a (ISO i mag e o r C D R O M) radio button.

Fig u re 7.3. T h e New VM win d o w - St ep 1

Click Fo rward to continue.

4. Select t h e in st allat io n med ia

Select the appropriate radio button for your installation media.

4 5

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

4 6

Fig u re 7.4 . Lo cat e yo u r in st all med ia

A. If you wish to install from a CD-ROM or DVD, select the Use C D R O M o r D VD radio button, and select the appropriate disk drive from the drop-down list of drives available.

B. If you wish to install from an ISO image, select Use ISO i mag e, and then click the

Bro wse. . . button to open the Lo cate med i a vo l ume window.

Select the installation image you wish to use, and click C ho o se Vo l ume.

If no images are displayed in the Lo cate med i a vo l ume window, click on the Bro wse

Lo cal button to browse the host machine for the installation image or DVD drive containing the installation disk. Select the installation image or DVD drive containing the installation disk and click O pen; the volume is selected for use and you are returned to the

C reate a new vi rtual machi ne wizard.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Important

For ISO image files and guest storage images, the recommended location to use is

/var/l i b/l i bvi rt/i mag es/. Any other location may require additional configuration by SELinux. Refer to the Red Hat Enterprise Linux 6 Virtualization

Administration Guide for more details on configuring SELinux.

Select the operating system type and version which match the installation media you have selected.

Fig u re 7.5. T h e New VM win d o w - St ep 2

Click Fo rward to continue.

5. Set RAM an d virt u al CPUs

Choose appropriate values for the virtual CPUs and RAM allocation. These values affect the host's and guest's performance. Memory and virtual CPUs can be overcommitted. For more information on overcommitting, refer to the Red Hat Enterprise Linux 6 Virtualization Administration

Guide.

4 7

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Virtual machines require sufficient physical memory (RAM) to run efficiently and effectively.

Red Hat supports a minimum of 512MB of RAM for a virtual machine. Red Hat recommends at least 1024MB of RAM for each logical core.

Assign sufficient virtual CPUs for the virtual machine. If the virtual machine runs a multithreaded application, assign the number of virtual CPUs the guest virtual machine will require to run efficiently.

You cannot assign more virtual CPUs than there are physical processors (or hyper-threads) available on the host system. The number of virtual CPUs available is noted in the Up to X

avai l abl e field.

4 8

Fig u re 7.6 . T h e n ew VM win d o w - St ep 3

Click Fo rward to continue.

6. St o rag e

Enable and assign storage for the Red Hat Enterprise Linux 6 guest virtual machine. Assign at least 5GB for a desktop installation or at least 1GB for a minimal installation.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Note

Live and offline migrations require virtual machines to be installed on shared network storage. For information on setting up shared storage for virtual machines, refer to the

Red Hat Enterprise Linux Virtualization Administration Guide.

a. Wit h t h e d ef au lt lo cal st o rag e

Select the C reate a d i sk i mag e o n the co mputer' s hard d ri ve radio button to create a file-based image in the default storage pool, the

/var/l i b/l i bvi rt/i mag es/ directory. Enter the size of the disk image to be created. If the Al l o cate enti re d i sk no w check box is selected, a disk image of the size specified will be created immediately. If not, the disk image will grow as it becomes filled.

Note

Although the storage pool is a virtual container it is limited by two factors: maximum size allowed to it by qemu-kvm and the size of the disk on the host physical machine. Storage pools may not exceed the size of the disk on the host physical machine. The maximum sizes are as follows: virtio-blk = 2^63 bytes or 8 Exabytes(using raw files or disk)

Ext4 = ~ 16 TB (using 4 KB block size)

XFS = ~8 Exabytes qcow2 and host file systems keep their own metadata and scalability should be evaluated/tuned when trying very large image sizes. Using raw disks means fewer layers that could affect scalability or max size.

4 9

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 7.7. T h e New VM win d o w - St ep 4

Click Fo rward to create a disk image on the local hard drive. Alternatively, select

Sel ect manag ed o r o ther exi sti ng sto rag e, then select Bro wse to configure managed storage.

b. Wit h a st o rag e p o o l

If you selected Sel ect manag ed o r o ther exi sti ng sto rag e in the previous step to use a storage pool and clicked Bro wse, the Lo cate o r create sto rag e

vo l ume window will appear.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Fig u re 7.8. T h e Lo cat e o r creat e st o rag e vo lu me win d o w i. Select a storage pool from the Sto rag e P o o l s list.

ii. Optional: Click on the New Vo l ume button to create a new storage volume.

The Ad d a Sto rag e Vo l ume screen will appear. Enter the name of the new storage volume.

Choose a format option from the Fo rmat dropdown menu. Format options include raw, cow, qcow, qcow2, qed, vmdk, and vpc. Adjust other fields as desired.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 7.9 . T h e Ad d a St o rag e Vo lu me win d o w

Click Fi ni sh to continue.

7. Verif y an d f in ish

Verify there were no errors made during the wizard and everything appears as expected.

Select the C usto mi ze co nfi g urati o n befo re i nstal l check box to change the guest's storage or network devices, to use the para-virtualized drivers or to add additional devices.

Click on the Ad vanced o pti o ns down arrow to inspect and modify advanced options. For a standard Red Hat Enterprise Linux 6 installation, none of these options require modification.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Fig u re 7.10. T h e New VM win d o w - lo cal st o rag e

Click Fi ni sh to continue into the Red Hat Enterprise Linux installation sequence. For more information on installing Red Hat Enterprise Linux 6 refer to the Red Hat Enterprise Linux 6

Installation Guide.

A Red Hat Enterprise Linux 6 guest virtual machine is now created from an ISO installation disc image.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

7.2. Creat ing a Red Hat Ent erprise Linux 6 guest wit h a net work inst allat ion t ree

Pro ced u re 7.2. Creat in g a Red Hat En t erp rise Lin u x 6 g u est wit h virt - man ag er

1. O p t io n al: Prep arat io n

Prepare the storage environment for the guest virtual machine. For more information on preparing storage, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.

Important

Various storage types may be used for storing guest virtual machines. However, for a virtual machine to be able to use migration features the virtual machine must be created on networked storage.

2. O p en virt - man ag er an d st art t h e wiz ard

Open virt-manager by executing the vi rt-manag er command as root or opening

Appl i cati o ns → System T o o l s → Vi rtual Machi ne Manag er.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Fig u re 7.11. T h e main virt - man ag er win d o w

Click on the C reate a new vi rtual machi ne button to start the new virtual machine wizard.

Fig u re 7.12. T h e Creat e a n ew virt u al mach in e b u t t o n

The C reate a new vi rtual machi ne window opens.

3. Name t h e virt u al mach in e

Virtual machine names can contain letters, numbers and the following characters: '_', '. ' and

'-'. Virtual machine names must be unique for migration and cannot consist only of numbers.

Choose the installation method from the list of radio buttons.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 7.13. T h e New VM win d o w - St ep 1

Click Fo rward to continue.

4. Provide the installation URL, and the Kickstart URL and Kernel options if required.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Fig u re 7.14 . T h e New VM win d o w - St ep 2

Click Fo rward to continue.

5. The remaining steps are the same as the ISO installation procedure. Continue from Step 5 of the ISO installation procedure.

7.3. Creat ing a Red Hat Ent erprise Linux 6 guest wit h PXE

Pro ced u re 7.3. Creat in g a Red Hat En t erp rise Lin u x 6 g u est wit h virt - man ag er

1. O p t io n al: Prep arat io n

Prepare the storage environment for the virtual machine. For more information on preparing storage, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Important

Various storage types may be used for storing guest virtual machines. However, for a virtual machine to be able to use migration features the virtual machine must be created on networked storage.

2. O p en virt - man ag er an d st art t h e wiz ard

Open virt-manager by executing the vi rt-manag er command as root or opening

Appl i cati o ns → System T o o l s → Vi rtual Machi ne Manag er.

Fig u re 7.15. T h e main virt - man ag er win d o w

Click on the C reate new vi rtual i zed g uest button to start the new virtualized guest wizard.

⁠Chapt er 7 . Inst alling a Red Hat Ent erprise Linux 6 guest virt ual machine on a Red Hat Ent erprise Linux 6 host

Fig u re 7.16 . T h e creat e n ew virt u aliz ed g u est b u t t o n

The New VM window opens.

3. Name t h e virt u al mach in e

Virtual machine names can contain letters, numbers and the following characters: '_', '. ' and

'-'. Virtual machine names must be unique for migration and cannot consist only of numbers.

Choose the installation method from the list of radio buttons.

Fig u re 7.17. T h e New VM win d o w - St ep 1

Click Fo rward to continue.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

4. The remaining steps are the same as the ISO installation procedure. Continue from Step 5 of the ISO installation procedure. From this point, the only difference in this PXE procedure is on the final New VM screen, which shows the Instal l : P XE Instal l field.

Fig u re 7.18. T h e New VM win d o w - St ep 5 - PXE In st all

⁠Chapt er 8 . Virt ualizing Red Hat Ent erprise Linux on O t her Plat forms

Chapter 8. Virtualizing Red Hat Enterprise Linux on Other

Platforms

This chapter contains useful reference material for customers running Red Hat Enterprise Linux 6 as a virtualized operating system on other virtualization hosts.

8.1. On VMware ESX

Red Hat Enterprise Linux 6.0 and onward provide the vmw_bal l o o n driver, a para-virtualized memory ballooning driver used when running Red Hat Enterprise Linux on VMware hosts. For further information about this driver, refer to http://kb.VMware.com/selfservice/microsites/search.do?

cmd=displayKC&docType=kc&externalId=1002586 .

Red Hat Enterprise Linux 6.3 and onward provide the vmmo use_d rv driver, a para-virtualized mouse driver used when running Red Hat Enterprise Linux on VMware hosts. For further information about this driver, refer to http://kb.VMware.com/selfservice/microsites/search.do?

cmd=displayKC&docType=kc&externalId=5739104 .

Red Hat Enterprise Linux 6.3 and onward provide the vmware_d rv driver, a para-virtualized video driver used when running Red Hat Enterprise Linux on VMware hosts. For further information about this driver, refer to http://kb.VMware.com/selfservice/microsites/search.do?

cmd=displayKC&docType=kc&externalId=1033557 .

Red Hat Enterprise Linux 6.3 and onward provide the vmxnet3 driver, a para-virtualized network adapter used when running Red Hat Enterprise Linux on VMware hosts. For further information about this driver, refer to http://kb.VMware.com/selfservice/microsites/search.do?

language=en_US&cmd=displayKC&externalId=1001805 .

Red Hat Enterprise Linux 6.4 and onward provide the vmw_pvscsi driver, a para-virtualized SCSI adapter used when running Red Hat Enterprise Linux on VMware hosts. For further information about this driver, refer to http://kb.VMware.com/selfservice/microsites/search.do?

language=en_US&cmd=displayKC&externalId=1010398 .

8.2. On Hyper-V

Red Hat Enterprise Linux 6.4 and onward ships with a Microsoft's Linux Integration Services, a set of drivers that enable synthetic device support in supported virtualized operating systems. Further details about the drivers provided are available from http://technet.microsoft.com/enus/library/dn531030.aspx

Important

Note that the built-in Red Hat Enterprise Linux Integration Service drivers for Hyper-V are sufficient for Red Hat Enterprise Linux guests to run using the high performance synthetic devices on Hyper-V hosts. These built-in drivers are certified by Red Hat for this use, and certified configurations can be viewed on the Red Hat Customer Portal .

Therefore, it is not necessary to download and install Linux Integration Services (LIS) packages, and doing so may limit your Red Hat support, as described in the related

Knowledgebase article .

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

The following enhancements have been made to allow for easier deployment and management of

Red Hat Enterprise Linux guest virtual machines:

Upgraded VMBUS protocols - VMBUS protocols have been upgraded to Windows 8 level. As part of this work, now VMBUS interrupts can be processed on all available virtual CPUs in the guest.

Furthermore, the signaling protocol between the Red Hat Enterprise Linux guest virtual machine and the Windows host physical machine has been optimized.

Synthetic frame buffer driver - Provides enhanced graphics performance and superior resolution for Red Hat Enterprise Linux desktop users.

Live Virtual Machine Backup support - Provisions uninterrupted backup support for live Red Hat

Enterprise Linux guest virtual machines.

Dynamic expansion of fixed size Linux VHDXs - Allows expansion of live mounted fixed size Red

Hat Enterprise Linux VHDXs.

Boot using UEFI - Allows virtual machines to boot using Unified Extensible Firmware Interface

(UEFI) on a Hyper-V 2012 R2 host.

For more information and features, refer to Red Hat Enterprise Linux — Virtualization Administration Guide and to the following article:" Enabling Linux Support on Windows Server 2012 R2 Hyper-V ".

⁠Chapt er 9 . Inst alling a fully- virt ualized Windows guest

Chapter 9. Installing a fully-virtualized Windows guest

This chapter describes how to create a fully-virtualized Windows guest using the command-line

(vi rt-i nstal l ), launch the operating system's installer inside the guest, and access the installer through vi rt-vi ewer.

To install a Windows operating system on the guest, use the vi rt-vi ewer tool. This tool allows you to display the graphical console of a virtual machine (via the spice or VNC protocol). In doing so,

vi rt-vi ewer allows you to install a fully-virtualized guest's operating system with that operating system's installer .

Installing a Windows operating system involves two major steps:

1. Creating the guest virtual machine, using either vi rt-i nstal l or vi rt-manag er.

2. Installing the Windows operating system on the guest virtual machine, using vi rt-vi ewer.

Refer to

Chapter 6, Guest virtual machine installation overview for details about creating a guest virtual

machine with vi rt-i nstal l or vi rt-manag er.

Note that this chapter does not describe how to install a Windows operating system on a fullyvirtualized guest. Rather, it only covers how to create the guest and launch the installer within the guest. For information on how to install a Windows operating system, refer to the relevant Microsoft installation documentation.

9.1. Using virt -inst all t o creat e a guest

The vi rt-i nstal l command allows you to create a fully-virtualized guest from a terminal, for example, without a GUI.

Important

Before creating the guest, consider first if the guest needs to use KVM Windows paravirtualized drivers. If it does, keep in mind that you can do so during or after installing the

Windows operating system on the guest. For more information about para-virtualized drivers,

refer to Chapter 10, KVM Para-virtualized (virtio) Drivers

For instructions on how to install KVM para-virtualized drivers, refer to Section 10.1, “ Installing the KVM Windows virtio drivers” .

It is possible to create a fully-virtualized guest with only a single command. To do so, run the following program (replace the values accordingly):

# virt-install \

--name=guest-name \

--os-type=windows \

--network network=default \

--disk path=path-to-disk,size=disk-size \

--cdrom=path-to-install-disk \

--graphics spice --ram=1024

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

The path-to-disk must be a device (e.g. /d ev/sd a3) or image file

(/var/l i b/l i bvi rt/i mag es/name. i mg ). It must also have enough free space to support the

disk-size.

Important

All image files are stored in /var/l i b/l i bvi rt/i mag es/ by default. Other directory locations for file-based images are possible, but may require SELinux configuration. If you run

SELinux in enforcing mode, refer to the Red Hat Enterprise Linux 6 Virtualization Administration

Guide for more information on SELinux.

You can also run vi rt-i nstal l interactively. To do so, use the --pro mpt command, as in:

# virt-install --prompt

Once the fully-virtualized guest is created, vi rt-vi ewer will launch the guest and run the operating system's installer. Refer to the relevant Microsoft installation documentation for instructions on how to install the operating system.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Chapter 10. KVM Para-virtualized (virtio) Drivers

Para-virtualized drivers enhance the performance of guests, decreasing guest I/O latency and increasing throughput to near bare-metal levels. It is recommended to use the para-virtualized drivers for fully virtualized guests running I/O heavy tasks and applications.

Virtio drivers are KVM's para-virtualized device drivers, available for Windows guest virtual machines running on KVM hosts. These drivers are included in the virtio package. The virtio package supports block (storage) devices and network interface controllers.

The KVM virtio drivers are automatically loaded and installed on the following:

Red Hat Enterprise Linux 4.8 and newer

Red Hat Enterprise Linux 5.3 and newer

Red Hat Enterprise Linux 6 and newer

Red Hat Enterprise Linux 7 and newer

Some versions of Linux based on the 2.6.27 kernel or newer kernel versions.

Versions of Red Hat Enterprise Linux in the list above detect and install the drivers, additional installation steps are not required.

In Red Hat Enterprise Linux 3 (3.9 and above), manual installation is required.

Note

PCI devices are limited by the virtualized system architecture. Refer to Section 4.1, “ KVM restrictions” for additional limitations when using assigned devices.

Using KVM virtio drivers, the following Microsoft Windows versions are expected to run similarly to bare-metal-based systems.

Windows Server 2003 (32-bit and 64-bit versions)

Windows Server 2008 (32-bit and 64-bit versions)

Windows Server 2008 R2 (64-bit only)

Windows 7 (32-bit and 64-bit versions)

Windows Server 2012 (64-bit only)

Windows Server 2012 R2 (64-bit only)

Windows 8 (32-bit and 64-bit versions)

Windows 8.1 (32-bit and 64-bit versions)

10.1. Inst alling t he KVM Windows virt io drivers

This section covers the installation process for the KVM Windows virtio drivers. The KVM virtio drivers can be loaded during the Windows installation or installed after the guest is installed.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

You can install the virtio drivers on a guest virtual machine using one of the following methods: hosting the installation files on a network accessible to the virtual machine using a virtualized CD-ROM device of the driver installation disk .iso file using a USB drive, by mounting the same (provided) .ISO file that you would use for the CD-ROM using a virtualized floppy device to install the drivers during boot time (required and recommended only for XP/2003)

This guide describes installation from the para-virtualized installer disk as a virtualized CD-ROM device.

1. Do wn lo ad t h e d rivers

The virtio-win package contains the virtio block and network drivers for all supported

Windows guest virtual machines.

Download and install the virtio-win package on the host with the yum command.

# yum install virtio-win

The list of virtio-win packages that are supported on Windows operating systems, and the current certified package version, can be found at the following URL: windowsservercatalog.com

Note that the Red Hat Enterprise Virtualization Hypervisor and Red Hat Enterprise Linux are created on the same code base so the drivers for the same version (for example, Red Hat

Enterprise Virtualization Hypervisor 3.3 and Red Hat Enterprise Linux 6.5) are supported for both environments.

The virtio-win package installs a CD-ROM image, vi rti o -wi n. i so , in the

/usr/share/vi rti o -wi n/ directory.

2. In st all t h e virt io d rivers

When booting a Windows guest that uses virtio-win devices, the relevant virtio-win device drivers must already be installed on this guest. The virtio-win drivers are not provided as inbox drivers in Microsoft's Windows installation kit, so installation of a Windows guest on a virtio-win storage device (viostor/virtio-scsi) requires that you provide the appropriate driver

during the installation, either directly from the vi rti o -wi n. i so or from the supplied Virtual

Floppy image vi rti o -wi n<version>. vfd .

10.2. Inst alling t he drivers on an inst alled Windows guest virt ual machine

This procedure covers installing the virtio drivers with a virtualized CD-ROM after Windows is installed.

Follow this procedure to add a CD-ROM image with vi rt-manag er and then install the drivers.

Pro ced u re 10.1. In st allin g f ro m t h e d river CD- RO M imag e wit h virt - man ag er

1. O p en virt - man ag er an d t h e g u est virt u al mach in e

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Open vi rt-manag er, then open the guest virtual machine from the list by double-clicking the guest name.

2. O p en t h e h ard ware win d o w

Click the lightbulb icon on the toolbar at the top of the window to view virtual hardware details.

Fig u re 10.1. T h e virt u al h ard ware d et ails b u t t o n

Then click the Ad d Hard ware button at the bottom of the new view that appears. This opens a wizard for adding the new device.

3. Select t h e d evice t yp e — f o r Red Hat En t erp rise Lin u x 6 versio n s p rio r t o 6 .2

Skip this step if you are using Red Hat Enterprise Linux 6.2 or later.

On Red Hat Enterprise Linux 6 versions prior to version 6.2, you must select the type of device you wish to add. In this case, select Sto rag e from the dropdown menu.

Fig u re 10.2. T h e Ad d n ew virt u al h ard ware wiz ard in Red Hat En t erp rise Lin u x 6 .1

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Click the Fi ni sh button to proceed.

4. Select t h e ISO f ile

Ensure that the Sel ect manag ed o r o ther exi sti ng sto rag e radio button is selected, and browse to the virtio driver's . i so image file. The default location for the latest version of the drivers is /usr/share/vi rti o -wi n/vi rti o -wi n. i so .

Change the D evi ce type to ID E cd ro m and click the Fo rward button to proceed.

Fig u re 10.3. T h e Ad d n ew virt u al h ard ware wiz ard

5. Fin ish ad d in g virt u al h ard ware — f o r Red Hat En t erp rise Lin u x 6 versio n s p rio r t o

6 .2

If you are using Red Hat Enterprise Linux 6.2 or later, skip this step.

On Red Hat Enterprise Linux 6 versions prior to version 6.2, click on the Fi ni sh button to finish adding the virtual hardware and close the wizard.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.4 . T h e Ad d n ew virt u al h ard ware wiz ard in Red Hat En t erp rise Lin u x 6 .1

6. Reb o o t

Reboot or start the virtual machine to begin using the driver disc. Virtualized IDE devices require a restart to for the virtual machine to recognize the new device.

Once the CD-ROM with the drivers is attached and the virtual machine has started, proceed with

Procedure 10.2, “ Windows installation on a Windows 7 virtual machine”

Pro ced u re 10.2. Win d o ws in st allat io n o n a Win d o ws 7 virt u al mach in e

This procedure installs the drivers on a Windows 7 virtual machine as an example. Adapt the

Windows installation instructions to your guest's version of Windows.

1. O p en t h e Co mp u t er Man ag emen t win d o w

On the desktop of the Windows virtual machine, click the Wi nd o ws icon at the bottom corner of the screen to open the Start menu.

Right-click on C o mputer and select Manag e from the pop-up menu.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.5. T h e Co mp u t er Man ag emen t win d o w

2. O p en t h e Device Man ag er

Select the D evi ce Manag er from the left-most pane. This can be found under C o mputer

Manag ement > System T o o l s.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.6 . T h e Co mp u t er Man ag emen t win d o w

3. St art t h e d river u p d at e wiz ard a. View availab le syst em d evices

Expand System d evi ces by clicking on the arrow to its left.

Fig u re 10.7. Viewin g availab le syst em d evices in t h e Co mp u t er

Man ag emen t win d o w b. Lo cat e t h e ap p ro p riat e d evice

There are up to four drivers available: the balloon driver, the serial driver, the network driver, and the block driver.

Bal l o o n, the balloon driver, affects the P C I stand ard R AM C o ntro l l er in the System d evi ces group.

vi o seri al , the serial driver, affects the P C I Si mpl e C o mmuni cati o n

C o ntro l l er in the System d evi ces group.

NetKVM, the network driver, affects the Netwo rk ad apters group. This driver is only available if a virtio NIC is configured. Configurable parameters for this driver are documented in

Appendix A, NetKVM Driver Parameters

vi o sto r, the block driver, affects the D i sk d ri ves group. This driver is only available if a virtio disk is configured.

Right-click on the device whose driver you wish to update, and select Upd ate

D ri ver. . . from the pop-up menu.

This example installs the balloon driver, so right-click on P C I stand ard R AM

C o ntro l l er.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.8. T h e Co mp u t er Man ag emen t win d o w c. O p en t h e d river u p d at e wiz ard

From the drop-down menu, select Upd ate D ri ver So ftware. . . to access the driver update wizard.

Fig u re 10.9 . O p en in g t h e d river u p d at e wiz ard

4. Sp ecif y h o w t o f in d t h e d river

The first page of the driver update wizard asks how you want to search for driver software.

Click on the second option, Bro wse my co mputer fo r d ri ver so ftware.

Fig u re 10.10. T h e d river u p d at e wiz ard

5. Select t h e d river t o in st all a. O p en a f ile b ro wser

Click on Bro wse. . .

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.11. T h e d river u p d at e wiz ard b. Bro wse t o t h e lo cat io n o f t h e d river

A separate driver is provided for each of the various combinations of operating system and architecture. The drivers are arranged hierarchically according to their driver type, the operating system, and the architecture on which they will be installed:

driver_type/os/arch/. For example, the Balloon driver for a Windows 7 operating system with an x86 (32-bit) architecture, resides in the Bal l o o n/w7/x86 directory.

Fig u re 10.12. T h e Bro wse f o r d river so f t ware p o p - u p win d o w

Once you have navigated to the correct location, click O K.

c. Click Next t o co n t in u e

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.13. T h e Up d at e Driver So f t ware wiz ard

The following screen is displayed while the driver installs:

Fig u re 10.14 . T h e Up d at e Driver So f t ware wiz ard

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

6. Clo se t h e in st aller

The following screen is displayed when installation is complete:

Fig u re 10.15. T h e Up d at e Driver So f t ware wiz ard

Click C l o se to close the installer.

7. Reb o o t

Reboot the virtual machine to complete the driver installation.

10.3. Inst alling drivers during t he Windows inst allat ion

This procedure covers installing the virtio drivers during a Windows installation.

This method allows a Windows guest virtual machine to use the virtio drivers for the default storage device.

Pro ced u re 10.3. In st allin g virt io d rivers d u rin g t h e Win d o ws in st allat io n

1. Install the virtio-win package:

# yum install virtio-win

2. Creat in g t h e g u est virt u al mach in e

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Important

Create the virtual machine, as normal, without starting the virtual machine. Follow one of the procedures below.

Select one of the following guest-creation methods, and follow the instructions.

a. Creat in g t h e g u est virt u al mach in e wit h virsh

This method attaches the virtio driver floppy disk to a Windows guest before the installation.

If the virtual machine is created from an XML definition file with vi rsh, use the vi rsh

d efi ne command not the vi rsh create command.

i. Create, but do not start, the virtual machine. Refer to the Red Hat Enterprise

Linux Virtualization Administration Guide for details on creating virtual machines with the vi rsh command.

ii. Add the driver disk as a virtualized floppy disk with the vi rsh command. This example can be copied and used if there are no other virtualized floppy devices attached to the guest virtual machine. Note that vm_name should be replaced with the name of the virtual machine.

# virsh attach-disk vm_name /usr/share/virtiowin/virtio-win.vfd fda --type floppy

You can now continue with Step 3

b. Creat in g t h e g u est virt u al mach in e wit h virt - man ag er an d ch an g in g t h e d isk t yp e i. At the final step of the virt-manager guest creation wizard, check the

C usto mi ze co nfi g urati o n befo re i nstal l checkbox.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.16 . T h e virt - man ag er g u est creat io n wiz ard

Click on the Fi ni sh button to continue.

ii. O p en t h e Ad d Hard ware wiz ard

Click the Ad d Hard ware button in the bottom left of the new panel.

iii. Select st o rag e d evice

Sto rag e is the default selection in the Hard ware type list.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.17. T h e Ad d n ew virt u al h ard ware wiz ard

Ensure the Sel ect manag ed o r o ther exi sti ng sto rag e radio button is selected. Click Bro wse. . . .

Fig u re 10.18. Select man ag ed o r exist in g st o rag e

In the new window that opens, click Bro wse Lo cal . Navigate to

/usr/share/vi rti o -wi n/vi rti o -wi n. vfd , and click Sel ect to confirm.

Change D evi ce type to Fl o ppy d i sk, and click Fi ni sh to continue.

Fig u re 10.19 . Ch an g e t h e Device t yp e iv. Co n f irm set t in g s

Review the device settings.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.20. T h e virt u al mach in e h ard ware in f o rmat io n win d o w

You have now created a removable device accessible by your virtual machine.

v. Ch an g e t h e h ard d isk t yp e

To change the hard disk type from IDE Disk to Virtio Disk, we must first remove the existing hard disk, Disk 1. Select the disk and click on the R emo ve button.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.21. T h e virt u al mach in e h ard ware in f o rmat io n win d o w

Add a new virtual storage device by clicking Ad d Hard ware. Then, change the D evi ce type from IDE disk to Virtio Disk. Click Fi ni sh to confirm the operation.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.22. T h e virt u al mach in e h ard ware in f o rmat io n win d o w vi. En su re set t in g s are co rrect

Review the settings for VirtIO Disk 1.

Fig u re 10.23. T h e virt u al mach in e h ard ware in f o rmat io n win d o w

When you are satisfied with the configuration details, click the Beg i n

Instal l ati o n button.

You can now continue with Step 3

c. Creat in g t h e g u est virt u al mach in e wit h virt - in st all

Append the following parameter exactly as listed below to add the driver disk to the installation with the vi rt-i nstal l command:

--disk path=/usr/share/virtio-win/virtiowin.vfd,device=floppy

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Important

If the device you wish to add is a d i sk (that is, not a fl o ppy or a cd ro m), you will also need to add the bus= vi rti o option to the end of the --disk parameter, like so:

--disk path=/usr/share/virtio-win/virtiowin.vfd,device=disk,bus=virtio

According to the version of Windows you are installing, append one of the following options to the vi rt-i nstal l command:

--os-variant win2k3

--os-variant win7

You can now continue with

Step 3 .

Ad d it io n al st ep s f o r d river in st allat io n

During the installation, additional steps are required to install drivers, depending on the type of Windows guest.

Win d o ws Server 2003

Before the installation blue screen repeatedly press F6 for third party drivers.

Fig u re 10.24 . T h e Win d o ws Set u p screen

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Press S to install additional device drivers.

Fig u re 10.25. T h e Win d o ws Set u p screen

Fig u re 10.26 . T h e Win d o ws Set u p screen b.

Press Enter to continue the installation.

Win d o ws Server 2008

Follow the same procedure for Windows Server 2003, but when the installer prompts you for the driver, click on Lo ad D ri ver, point the installer to D ri ve A: and pick the driver that suits your guest operating system and architecture.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

10.4 . Using t he virt io drivers wit h Red Hat Ent erprise Linux 3.9 guest s

Virtio drivers for Red Hat Enterprise Linux 3.9 consist of five kernel modules: vi rti o , vi rti o _bl k,

vi rti o _net, vi rti o _pci and vi rti o _ri ng . All five modules must be loaded to use both the virtio block and network devices drivers.

Important

For Red Hat Enterprise Linux 3.9 guests, the kmod-virtio package is a requirement for the

vi rti o module.

Note

To use the network device driver only, load the vi rti o , vi rti o _net and vi rti o _pci modules. To use the block device driver only, load the vi rti o , vi rti o _ri ng , vi rti o _bl k and vi rti o _pci modules.

Important

The virtio package modifies the initrd RAM disk file in the /bo o t directory. The original initrd file is saved to /bo o t/i ni trd -kernel-version. i mg . vi rti o . o ri g . The original initrd file is replaced with a new initrd RAM disk containing the vi rti o driver modules. The initrd

RAM disk is modified to allow the virtual machine to boot from a storage device using the virtio drivers. To use a different initrd file, you must ensure that drivers are loaded with the sysi ni t script (

Loading the virtio drivers with the sysinit script ) or when creating new initrd RAM disk

(

Adding the virtio drivers to the initrd RAM disk ).

Lo ad in g t h e virt io d rivers wit h t h e sysin it scrip t

This procedure covers loading the virtio driver modules during the boot sequence on a Red Hat

Enterprise Linux 3.9 or newer guest with the sysi ni t script. Note that the guest virtual machine cannot use the virtio drivers for the default boot disk if the modules are loaded with the sysi ni t script.

The drivers must be loaded in the following order:

1. vi rti o

2. vi rti o _ri ng

3. vi rti o _pci

4. vi rti o _bl k

5. vi rti o _net

vi rti o _net and vi rti o _bl k are the only drivers whose order can be changed. If other drivers are loaded in a different order, they will not work.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Next, configure the modules. Locate the following section of the /etc/rc. d /rc. sysi ni t file.

if [ -f /etc/rc.modules ]; then

/etc/rc.modules

Append the following lines after that section: if [ -f /etc/rc.modules ]; then

/etc/rc.modules

fi modprobe virtio modprobe virtio_ring # Comment this out if you do not need block driver modprobe virtio_blk # Comment this out if you do not need block driver modprobe virtio_net # Comment this out if you do not need net driver modprobe virtio_pci

Reboot the guest virtual machine to load the kernel modules.

Ad d in g t h e virt io d rivers t o t h e in it rd RAM d isk

This procedure covers loading the virtio driver modules with the kernel on a Red Hat Enterprise Linux

3.9 or newer guest by including the modules in the initrd RAM disk. The mkinitrd tool configures the initrd RAM disk to load the modules. Specify the additional modules with the --with parameter for the mki ni trd command. Append following set of parameters, in the exact order, when using the

mki ni trd command to create a custom initrd RAM disk:

--with virtio --with virtio_ring --with virtio_blk --with virtio_net -with virtio_pci

AMD6 4 an d In t el 6 4 issu es

Use the x86 _6 4 version of the virtio package for AMD64 systems.

Use the i a32e version of the virtio package for Intel 64 systems. Using the x86 _6 4 version of the

virtio may cause a 'Unreso l ved symbo l ' error during the boot sequence on Intel 64 systems.

Net wo rk p erf o rman ce issu es

If you experience low performance with the virtio network drivers, verify the setting for the GSO and

TSO features on the host system. The virtio network drivers require that the GSO and TSO options are disabled for optimal performance.

Verify the status of the GSO and TSO settings, use the command on the host (replacing interface with the network interface used by the guest):

# ethtool -k interface

Disable the GSO and TSO options with the following commands on the host:

# ethtool -K interface gso off

# ethtool -K interface tso off

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Virt io d river swap p art it io n issu e

After activating the virtio block device driver the swap partition may not be available. This issue is may be caused by a change in disk device name. To fix this issue, open the /etc/fstab file and locate the lines containing swap partitions, for example:

/dev/hda3 swap swap defaults 0 0

The virtio drivers use the /d ev/vd * naming convention, not the /d ev/hd * naming convention. To resolve this issue modify the incorrect swap entries in the /etc/fstab file to use the /d ev/vd * convention, for the example above:

/dev/vda3 swap swap defaults 0 0

Save the changes and reboot the guest virtual machine. The virtual machine should now have swap partitions.

10.5. Using KVM virt io drivers for exist ing devices

You can modify an existing hard disk device attached to the guest to use the vi rti o driver instead of the virtualized IDE driver. The example shown in this section edits libvirt configuration files. Note that the guest virtual machine does not need to be shut down to perform these steps, however the change will not be applied until the guest is completely shut down and rebooted.

Pro ced u re 10.4 . Usin g KVM virt io d rivers f o r exist in g d evices

1. Ensure that you have installed the appropriate driver (vi o sto r), as described in

Section 10.1, “ Installing the KVM Windows virtio drivers” , before continuing with this

procedure.

2. Run the vi rsh ed i t <g uestname> command as root to edit the XML configuration file for your device. For example, vi rsh ed i t g uest1. The configuration files are located in

/etc/l i bvi rt/q emu.

3. Below is a file-based block device using the virtualized IDE driver. This is a typical entry for a virtual machine not using the virtio drivers.

</disk>

4. Change the entry to use the virtio device by modifying the b u s= entry to vi rti o . Note that if the disk was previously IDE it will have a target similar to hda, hdb, or hdc and so on. When changing to b u s= virt io the target needs to be changed to vda, vdb, or vdc accordingly.

<target d ev= ' vd a' bus= ' vi rti o ' />

</disk>

5. Remove the ad d ress tag inside the d isk tags. This must be done for this procedure to work.

Libvirt will regenerate the ad d ress tag appropriately the next time the virtual machine is started.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Alternatively, vi rt-manag er, vi rsh attach-d i sk or vi rsh attach-i nterface can add a new device using the virtio drivers.

Refer to the libvirt website for more details on using Virtio: http://www.linux-kvm.org/page/Virtio

10.6. Using KVM virt io drivers for new devices

This procedure covers creating new devices using the KVM virtio drivers with vi rt-manag er.

Alternatively, the vi rsh attach-d i sk or vi rsh attach-i nterface commands can be used to attach devices using the virtio drivers.

Important

Ensure the drivers have been installed on the Windows guest before proceeding to install new devices. If the drivers are unavailable the device will not be recognized and will not work.

Pro ced u re 10.5. Ad d in g a st o rag e d evice u sin g t h e virt io st o rag e d river

1. Open the guest virtual machine by double clicking on the name of the guest in vi rt-

manag er.

2. Open the Sho w vi rtual hard ware d etai l s tab by clicking the l i g htbul b button.

Fig u re 10.27. T h e Sh o w virt u al h ard ware d et ails t ab

3. In the Sho w vi rtual hard ware d etai l s tab, click on the Ad d Hard ware button.

4. Select h ard ware t yp e

Select Sto rag e as the Hard ware type.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.28. T h e Ad d n ew virt u al h ard ware wiz ard

5. Select t h e st o rag e d evice an d d river

Create a new disk image or select a storage pool volume.

Set the D evi ce type to Vi rti o d i sk to use the virtio drivers.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.29 . T h e Ad d n ew virt u al h ard ware wiz ard

Click Fi ni sh to complete the procedure.

Pro ced u re 10.6 . Ad d in g a n et wo rk d evice u sin g t h e virt io n et wo rk d river

1. Open the guest virtual machine by double clicking on the name of the guest in vi rt-

manag er.

2. Open the Sho w vi rtual hard ware d etai l s tab by clicking the l i g htbul b button.

Fig u re 10.30. T h e Sh o w virt u al h ard ware d et ails t ab

3. In the Sho w vi rtual hard ware d etai l s tab, click on the Ad d Hard ware button.

4. Select h ard ware t yp e

Select Netwo rk as the Hard ware type.

⁠Chapt er 1 0 . KVM Para- virt ualized (virt io) Drivers

Fig u re 10.31. T h e Ad d n ew virt u al h ard ware wiz ard

5. Select t h e n et wo rk d evice an d d river

Set the D evi ce mo d el to vi rti o to use the virtio drivers. Choose the desired Ho st

d evi ce.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 10.32. T h e Ad d n ew virt u al h ard ware wiz ard

Click Fi ni sh to complete the procedure.

Once all new devices are added, reboot the virtual machine. Windows virtual machines may not recognize the devices until the guest is rebooted.

⁠Chapt er 1 1 . Net work configurat ion

Chapter 11. Network configuration

This chapter provides an introduction to the common networking configurations used by libvirt based guest virtual machines. For additional information, consult the libvirt network architecture documentation: http://libvirt.org/intro.html

Red Hat Enterprise Linux 6 supports the following networking setups for virtualization: virtual networks using Network Address Translation (NAT) directly allocated physical devices using PCI device assignment directly allocated virtual functions using PCIe SR-IOV bridged networks

You must enable NAT, network bridging or directly assign a PCI device to allow external hosts access to network services on guest virtual machines.

11.1. Net work Address T ranslat ion (NAT ) wit h libvirt

One of the most common methods for sharing network connections is to use Network Address

Translation (NAT) forwarding (also known as virtual networks).

Ho st co n f ig u rat io n

Every standard l i bvi rt installation provides NAT-based connectivity to virtual machines as the default virtual network. Verify that it is available with the vi rsh net-l i st --al l command.

# virsh net-list --all

Name State Autostart

----------------------------------------default active yes

If it is missing, the example XML configuration file can be reloaded and activated:

# virsh net-define /usr/share/libvirt/networks/default.xml

The default network is defined from /usr/share/l i bvi rt/netwo rks/d efaul t. xml

Mark the default network to automatically start:

# virsh net-autostart default

Network default marked as autostarted

Start the default network:

# virsh net-start default

Network default started

Once the l i bvi rt default network is running, you will see an isolated bridge device. This device does not have any physical interfaces added. The new device uses NAT and IP forwarding to connect to the physical network. Do not add new interfaces.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes

l i bvi rt adds i ptabl es rules which allow traffic to and from guest virtual machines attached to the vi rbr0 device in the INP UT , FO R WAR D , O UT P UT and P O ST R O UT ING chains. l i bvi rt then attempts to enable the i p_fo rward parameter. Some other applications may disable i p_fo rward , so the best option is to add the following to /etc/sysctl . co nf.

net.ipv4.ip_forward = 1

G u est virt u al mach in e co n f ig u rat io n

Once the host configuration is complete, a guest virtual machine can be connected to the virtual network based on its name. To connect a guest to the 'default' virtual network, the following could be used in the XML configuration file (such as /etc/l i bvi rtd /q emu/myg uest. xml ) for the guest:

</interface>

Note

Defining a MAC address is optional. If you do not define one, a MAC address is automatically generated and used as the MAC address of the bridge device used by the network. Manually setting the MAC address may be useful to maintain consistency or easy reference throughout your environment, or to avoid the very small chance of a conflict.

</interface>

11.2. Disabling vhost -net

The vho st-net module is a kernel-level backend for virtio networking that reduces virtualization overhead by moving virtio packet processing tasks out of user space (the qemu process) and into the kernel (the vho st-net driver). vhost-net is only available for virtio network interfaces. If the vhost-net kernel module is loaded, it is enabled by default for all virtio interfaces, but can be disabled in the interface configuration in the case that a particular workload experiences a degradation in performance when vhost-net is in use.

Specifically, when UDP traffic is sent from a host machine to a guest virtual machine on that host, performance degradation can occur if the guest virtual machine processes incoming data at a rate slower than the host machine sends it. In this situation, enabling vho st-net causes the UDP socket's receive buffer to overflow more quickly, which results in greater packet loss. It is therefore better to disable vho st-net in this situation to slow the traffic, and improve overall performance.

⁠Chapt er 1 1 . Net work configurat ion

To disable vho st-net, edit the  sub-element in the guest virtual machine's XML configuration file and define the network as follows:

...

...

</interface>

Setting the driver name to q emu forces packet processing into qemu user space, effectively disabling vhost-net for that interface.

11.3. Bridged net working wit h libvirt

Bridged networking (also known as physical device sharing) is used to dedicate a physical device to a virtual machine. Bridging is often used for more advanced setups and on servers with multiple network interfaces.

To create a bridge (br0 ) based on the eth0 interface, execute the following command on the host:

# virsh iface-bridge eth0 br0

Important

NetworkManager does not support bridging. NetworkManager must be disabled to use networking with the network scripts (located in the /etc/sysco nfi g /netwo rk-scri pts/ directory).

# chkconfig NetworkManager off

# chkconfig network on

# service NetworkManager stop

# service network start

If you do not want to disable Net wo rkMan ag er entirely, add "NM_CONTROLLED=no" to the

i fcfg -* network script being used for the bridge.

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Chapter 12. PCI device assignment

Red Hat Enterprise Linux 6 exposes three classes of device to its virtual machines:

Emulated devices are purely virtual devices that mimic real hardware, allowing unmodified guest operating systems to work with them using their standard in-box drivers.

Virtio devices are purely virtual devices designed to work optimally in a virtual machine. Virtio devices are similar to emulated devices, however, non-Linux virtual machines do not include the drivers they require by default. Virtualization management software like the Virtual Machine

Manager (virt - man ag er) and the Red Hat Enterprise Virtualization Hypervisor install these drivers automatically for supported non-Linux guest operating systems.

Assigned devices are physical devices that are exposed to the virtual machine. This method is also known as 'passthrough'. Device assignment allows virtual machines exclusive access to PCI devices for a range of tasks, and allows PCI devices to appear and behave as if they were physically attached to the guest operating system.

Device assignment is supported on PCI Express devices, except graphics cards. Parallel PCI devices may be supported as assigned devices, but they have severe limitations due to security and system configuration conflicts.

Red Hat Enterprise Linux 6 supports 32 PCI device slots per virtual machine, and 8 PCI functions per device slot. This gives a theoretical maximum of 256 configurable PCI functions per guest.

However, this theoretical maximum is subject to the following limitations:

Each virtual machine supports a maximum of 8 assigned device functions.

4 PCI device slots are configured with 5 emulated devices (two devices are in slot 1) by default.

However, users can explicitly remove 2 of the emulated devices that are configured by default if the guest operating system does not require them for operation (the video adapter device in slot 2; and the memory balloon driver device in the lowest available slot, usually slot 3). This gives users a supported functional maximum of 30 PCI device slots per virtual machine.

Red Hat Enterprise Linux 6.0 and newer supports hot plugging assigned PCI devices into virtual machines. However, PCI device hot plugging operates at the slot level and therefore does not support multi-function PCI devices. Multi-function PCI devices are recommended for static device configuration only.

Note

Red Hat Enterprise Linux 6.0 limited guest operating system driver access to a device's standard and extended configuration space. Limitations that were present in Red Hat

Enterprise Linux 6.0 were significantly reduced in Red Hat Enterprise Linux 6.1, and enable a much larger set of PCI Express devices to be successfully assigned to KVM guests.

Secure device assignment also requires interrupt remapping support. If a platform does not support interrupt remapping, device assignment will fail. To use device assignment without interrupt remapping support in a development environment, set the allow_unsafe_assigned_interrupts

KVM module parameter to 1.

PCI device assignment is only available on hardware platforms supporting either Intel VT-d or AMD

IOMMU. These Intel VT-d or AMD IOMMU specifications must be enabled in BIOS for PCI device assignment to function.

⁠Chapt er 1 2 . PCI device assignment

Pro ced u re 12.1. Prep arin g an In t el syst em f o r PCI d evice assig n men t

1. En ab le t h e In t el VT - d sp ecif icat io n s

The Intel VT-d specifications provide hardware support for directly assigning a physical device to a virtual machine. These specifications are required to use PCI device assignment with Red Hat Enterprise Linux.

The Intel VT-d specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default. The terms used to refer to these specifications can differ between manufacturers; consult your system manufacturer's documentation for the appropriate terms.

2. Act ivat e In t el VT - d in t h e kern el

Activate Intel VT-d in the kernel by adding the intel_iommu=on parameter to the kernel line in the /bo o t/g rub/g rub. co nf file.

The example below is a modified g rub. co nf file with Intel VT-d activated.

default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz

hiddenmenu title Red Hat Enterprise Linux Server (2.6.32-330.x86_645)

root (hd0,0)

kernel /vmlinuz-2.6.32-330.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb quiet i ntel _i o mmu= o n

initrd /initrd-2.6.32-330.x86_64.img

3. Read y t o u se

Reboot the system to enable the changes. Your system is now capable of PCI device assignment.

Pro ced u re 12.2. Prep arin g an AMD syst em f o r PCI d evice assig n men t

1. En ab le t h e AMD IO MMU sp ecif icat io n s

The AMD IOMMU specifications are required to use PCI device assignment in Red Hat

Enterprise Linux. These specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default.

2. En ab le IO MMU kern el su p p o rt

Append amd_iommu=on to the kernel command line in /bo o t/g rub/g rub. co nf so that

AMD IOMMU specifications are enabled at boot.

3. Read y t o u se

Reboot the system to enable the changes. Your system is now capable of PCI device assignment.

12.1. Assigning a PCI device wit h virsh

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

These steps cover assigning a PCI device to a virtual machine on a KVM hypervisor.

This example uses a PCIe network controller with the PCI identifier code, pci _0 0 0 0 _0 1_0 0 _0 , and a fully virtualized guest machine named guest1-rhel6-64.

Pro ced u re 12.3. Assig n in g a PCI d evice t o a g u est virt u al mach in e wit h virsh

1. Id en t if y t h e d evice

First, identify the PCI device designated for device assignment to the virtual machine. Use the

l spci command to list the available PCI devices. You can refine the output of l spci with

g rep.

This example uses the Ethernet controller highlighted in the following output:

# lspci | grep Ethernet

0 0 : 19 . 0 Ethernet co ntro l l er: Intel C o rpo rati o n 8256 7LM-2 G i g abi t

Netwo rk C o nnecti o n

01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

This Ethernet controller is shown with the short identifier 0 0 : 19 . 0 . We need to find out the full identifier used by vi rsh in order to assign this PCI device to a virtual machine.

To do so, combine the vi rsh no d ed ev-l i st command with the g rep command to list all devices of a particular type (pci ) that are attached to the host machine. Then look at the output for the string that maps to the short identifier of the device you wish to use.

This example highlights the string that maps to the Ethernet controller with the short identifier

0 0 : 19 . 0 . Note that the : and . characters are replaced with underscores in the full identifier.

⁠Chapt er 1 2 . PCI device assignment pci_0000_00_1d_2 pci_0000_00_1d_7 pci_0000_00_1e_0 pci_0000_00_1f_0 pci_0000_00_1f_2 pci_0000_00_1f_3 pci_0000_01_00_0 pci_0000_01_00_1 pci_0000_02_00_0 pci_0000_02_00_1 pci_0000_06_00_0 pci_0000_07_02_0 pci_0000_07_03_0

Record the PCI device number that maps to the device you want to use; this is required in other steps.

2. Review d evice in f o rmat io n

Information on the domain, bus, and function are available from output of the vi rsh

no d ed ev-d umpxml command: virsh nodedev-dumpxml pci_0000_00_19_0

<parent>computer</parent>

</driver>

<product id='0x1502'>82579LM Gigabit Network

Connection</product>

<vendor id='0x8086'>Intel Corporation</vendor>

</capability>

</device>

3. Det ermin e req u ired co n f ig u rat io n d et ails

Refer to the output from the vi rsh no d ed ev-d umpxml pci _0 0 0 0 _0 0 _19 _0 command for the values required for the configuration file.

Optionally, convert slot and function values to hexadecimal values (from decimal) to get the

PCI bus addresses. Append "0x" to the beginning of the output to tell the computer that the value is a hexadecimal number.

The example device has the following values: bus = 0, slot = 25 and function = 0. The decimal configuration uses those three values:

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide bus='0' slot='25' function='0'

If you want to convert to hexadecimal values, you can use the pri ntf utility to convert from decimal values, as shown in the following example:

$ printf %x 0

$ printf %x 25

$ printf %x 0

The example device would use the following hexadecimal values in the configuration file: bus='0x0' slot='0x19' function='0x0'

4. Ad d co n f ig u rat io n d et ails

Run vi rsh ed i t, specifying the virtual machine name, and add a device entry in the

<so urce> section to assign the PCI device to the guest virtual machine.

# virsh edit guest1-rhel6-64

</source>

</hostdev>

Alternately, run vi rsh attach-d evi ce, specifying the virtual machine name and the guest's XML file: virsh attach-device guest1-rhel6-64 fi l e. xml

5. Allo w d evice man ag emen t

Set an SELinux boolean to allow the management of the PCI device from the virtual machine:

# setsebool -P virt_use_sysfs 1

6. St art t h e virt u al mach in e

# virsh start guest1-rhel6-64

The PCI device should now be successfully assigned to the virtual machine, and accessible to the guest operating system.

12.2. Assigning a PCI device wit h virt -manager

100

⁠Chapt er 1 2 . PCI device assignment

PCI devices can be added to guest virtual machines using the graphical vi rt-manag er tool. The following procedure adds a Gigabit Ethernet controller to a guest virtual machine.

Pro ced u re 12.4 . Assig n in g a PCI d evice t o a g u est virt u al mach in e u sin g virt - man ag er

1. O p en t h e h ard ware set t in g s

Open the guest virtual machine and click the Ad d Hard ware button to add a new device to the virtual machine.

Fig u re 12.1. T h e virt u al mach in e h ard ware in f o rmat io n win d o w

2. Select a PCI d evice

Select PCI Ho st Device from the Hard ware list on the left.

Select an unused PCI device. Note that selecting PCI devices presently in use on the host causes errors. In this example, a spare 82576 network device is used. Click Fi ni sh to complete setup.

101

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Fig u re 12.2. T h e Ad d n ew virt u al h ard ware wiz ard

3. Ad d t h e n ew d evice

The setup is complete and the guest virtual machine now has direct access to the PCI device.

102

⁠Chapt er 1 2 . PCI device assignment

Fig u re 12.3. T h e virt u al mach in e h ard ware in f o rmat io n win d o w

12.3. Assigning a PCI device wit h virt -inst all

To use virt - in st all to assign a PCI device, use the --host-device parameter.

Pro ced u re 12.5. Assig n in g a PCI d evice t o a virt u al mach in e wit h virt - in st all

1. Id en t if y t h e d evice

Identify the PCI device designated for device assignment to the guest virtual machine.

# lspci | grep Ethernet

00:19.0 Ethernet controller: Intel Corporation 82567LM-2 Gigabit

Network Connection

01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

The vi rsh no d ed ev-l i st command lists all devices attached to the system, and identifies each PCI device with a string. To limit output to only PCI devices, run the following command:

103

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

# virsh nodedev-list --cap pci pci_0000_00_00_0 pci_0000_00_01_0 pci_0000_00_03_0 pci_0000_00_07_0 pci_0000_00_10_0 pci_0000_00_10_1 pci_0000_00_14_0 pci_0000_00_14_1 pci_0000_00_14_2 pci_0000_00_14_3 pci_0000_00_19_0 pci_0000_00_1a_0 pci_0000_00_1a_1 pci_0000_00_1a_2 pci_0000_00_1a_7 pci_0000_00_1b_0 pci_0000_00_1c_0 pci_0000_00_1c_1 pci_0000_00_1c_4 pci_0000_00_1d_0 pci_0000_00_1d_1 pci_0000_00_1d_2 pci_0000_00_1d_7 pci_0000_00_1e_0 pci_0000_00_1f_0 pci_0000_00_1f_2 pci_0000_00_1f_3 pci_0000_01_00_0 pci_0000_01_00_1 pci_0000_02_00_0 pci_0000_02_00_1 pci_0000_06_00_0 pci_0000_07_02_0 pci_0000_07_03_0

Record the PCI device number; the number is needed in other steps.

Information on the domain, bus and function are available from output of the vi rsh

no d ed ev-d umpxml command:

# virsh nodedev-dumpxml pci_0000_01_00_0

</driver>

<product id='0x10c9'>82576 Gigabit Network Connection</product>

<vendor id='0x8086'>Intel Corporation</vendor>

104

⁠Chapt er 1 2 . PCI device assignment

</capability>

</device>

2. Ad d t h e d evice

Use the PCI identifier output from the vi rsh no d ed ev command as the value for the --

host-device parameter.

virt-install \

--name=guest1-rhel6-64 \

--disk path=/var/lib/libvirt/images/guest1-rhel6-64.img,size=8 \

--nonsparse --graphics spice \

--vcpus=2 --ram=2048 \

--location=http://example1.com/installation_tree/RHEL6.1-Serverx86_64/os \

--nonetworks \

--os-type=linux \

--os-variant=rhel6 \

--host-device=pci_0000_01_00_0

3. Co mp let e t h e in st allat io n

Complete the guest installation. The PCI device should be attached to the guest.

12.4 . Det aching an assigned PCI device

When a host PCI device has been assigned to a guest machine, the host can no longer use the device. Read this section to learn how to detach the device from the guest with vi rsh or virt -

man ag er so it is available for host use.

Pro ced u re 12.6 . Det ach in g a PCI d evice f ro m a g u est wit h virsh

1. Det ach t h e d evice

Use the following command to detach the PCI device from the guest by removing it in the guest's XML file:

# virsh detach-device name_of_guest file.xml

2. Re- at t ach t h e d evice t o t h e h o st ( o p t io n al)

If the device is in managed mode, skip this step. The device will be returned to the host automatically.

If the device is not using managed mode, use the following command to re-attach the PCI device to the host machine:

# virsh nodedev-reattach device

For example, to re-attach the pci _0 0 0 0 _0 1_0 0 _0 device to the host:

105

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide virsh nodedev-reattach pci_0000_01_00_0

The device is now available for host use.

Pro ced u re 12.7. Det ach in g a PCI Device f ro m a g u est wit h virt - man ag er

1. O p en t h e virt u al h ard ware d et ails screen

In virt - man ag er, double-click on the virtual machine that contains the device. Select the

Sho w vi rtual hard ware d etai l s button to display a list of virtual hardware.

Fig u re 12.4 . T h e virt u al h ard ware d et ails b u t t o n

2. Select an d remo ve t h e d evice

Select the PCI device to be detached from the list of virtual devices in the left panel.

106

Fig u re 12.5. Select in g t h e PCI d evice t o b e d et ach ed

Click the R emo ve button to confirm. The device is now available for host use.

⁠Chapt er 1 3. SR- IO V

Chapter 13. SR-IOV

13.1. Int roduct ion

Developed by the PCI-SIG (PCI Special Interest Group), the Single Root I/O Virtualization (SR-IOV) specification is a standard for a type of PCI device assignment that can share a single device to multiple virtual machines. SR-IOV improves device performance for virtual machines.

Fig u re 13.1. Ho w SR- IO V wo rks

SR-IOV enables a Single Root Function (for example, a single Ethernet port), to appear as multiple, separate, physical devices. A physical device with SR-IOV capabilities can be configured to appear in the PCI configuration space as multiple functions. Each device has its own configuration space complete with Base Address Registers (BARs).

SR-IOV uses two PCI functions:

Physical Functions (PFs) are full PCIe devices that include the SR-IOV capabilities. Physical

Functions are discovered, managed, and configured as normal PCI devices. Physical Functions configure and manage the SR-IOV functionality by assigning Virtual Functions.

Virtual Functions (VFs) are simple PCIe functions that only process I/O. Each Virtual Function is derived from a Physical Function. The number of Virtual Functions a device may have is limited by the device hardware. A single Ethernet port, the Physical Device, may map to many Virtual

Functions that can be shared to virtual machines.

The hypervisor can map one or more Virtual Functions to a virtual machine. The Virtual Function's configuration space is then mapped to the configuration space presented to the guest.

Each Virtual Function can only be mapped to a single guest at a time, as Virtual Functions require real hardware resources. A virtual machine can have multiple Virtual Functions. A Virtual Function appears as a network card in the same way as a normal network card would appear to an operating system.

The SR-IOV drivers are implemented in the kernel. The core implementation is contained in the PCI subsystem, but there must also be driver support for both the Physical Function (PF) and Virtual

Function (VF) devices. An SR-IOV capable device can allocate VFs from a PF. The VFs appear as

107

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

PCI devices which are backed on the physical PCI device by resources such as queues and register sets.

Ad van t ag es o f SR- IO V

SR-IOV devices can share a single physical port with multiple virtual machines.

Virtual Functions have near-native performance and provide better performance than paravirtualized drivers and emulated access. Virtual Functions provide data protection between virtual machines on the same physical server as the data is managed and controlled by the hardware.

These features allow for increased virtual machine density on hosts within a data center.

SR-IOV is better able to utilize the bandwidth of devices with multiple guests.

13.2. Using SR-IOV

This section covers the use of PCI passthrough to assign a Virtual Function of an SR-IOV capable multiport network card to a virtual machine as a network device.

SR-IOV Virtual Functions (VFs) can be assigned to virtual machines by adding a device entry in

<hostdev> with the vi rsh ed i t or vi rsh attach-d evi ce command. However, this can be problematic because unlike a regular network device, an SR-IOV VF network device does not have a permanent unique MAC address, and is assigned a new MAC address each time the host is rebooted.

Because of this, even if the guest is assigned the same VF after a reboot, when the host is rebooted the guest determines its network adapter to have a new MAC address. As a result, the guest believes there is new hardware connected each time, and will usually require re-configuration of the guest's network settings.

libvirt-0.9.10 and later contains the <interface type='hostdev'> interface device. Using this interface device, lib virt will first perform any network-specific hardware/switch initialization indicated

(such as setting the MAC address, VLAN tag, or 802.1Qbh virtualport parameters), then perform the

PCI device assignment to the guest.

Using the <interface type='hostdev'> interface device requires: an SR-IOV-capable network card, host hardware that supports either the Intel VT-d or the AMD IOMMU extensions, and the PCI address of the VF to be assigned.

For a list of network interface cards (NICs) with SR-IOV support, see https://access.redhat.com/articles/1390483 .

Important

Assignment of an SR-IOV device to a virtual machine requires that the host hardware supports the Intel VT-d or the AMD IOMMU specification.

To attach an SR-IOV network device on an Intel or an AMD system, follow this procedure:

Pro ced u re 13.1. At t ach an SR- IO V n et wo rk d evice o n an In t el o r AMD syst em

108

⁠Chapt er 1 3. SR- IO V

1. En ab le In t el VT - d o r t h e AMD IO MMU sp ecif icat io n s in t h e BIO S an d kern el

On an Intel system, enable Intel VT-d in the BIOS if it is not enabled already. Refer to

Procedure 12.1, “ Preparing an Intel system for PCI device assignment”

for procedural help on enabling Intel VT-d in the BIOS and kernel.

Skip this step if Intel VT-d is already enabled and working.

On an AMD system, enable the AMD IOMMU specifications in the BIOS if they are not enabled already. Refer to

Procedure 12.2, “ Preparing an AMD system for PCI device assignment”

for procedural help on enabling IOMMU in the BIOS.

2. Verif y su p p o rt

Verify if the PCI device with SR-IOV capabilities is detected. This example lists an Intel 82576 network interface card which supports SR-IOV. Use the l spci command to verify whether the device was detected.

# lspci

03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

Note that the output has been modified to remove all other devices.

3. St art t h e SR- IO V kern el mo d u les

If the device is supported the driver kernel module should be loaded automatically by the kernel. Optional parameters can be passed to the module using the mo d pro be command.

The Intel 82576 network interface card uses the i g b driver kernel module.

# modprobe igb [<option>=<VAL1>,<VAL2>,]

# lsmod |grep igb igb 87592 0 dca 6708 1 igb

4. Act ivat e Virt u al Fu n ct io n s

The max_vfs parameter of the i g b module allocates the maximum number of Virtual

Functions. The max_vfs parameter causes the driver to spawn, up to the value of the parameter in, Virtual Functions. For this particular card the valid range is 0 to 7.

Remove the module to change the variable.

# modprobe -r igb

Restart the module with the max_vfs set to 7 or any number of Virtual Functions up to the maximum supported by your device.

# modprobe igb max_vfs=7

5. Make t h e Virt u al Fu n ct io n s p ersist en t

109

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Add the line o pti o ns i g b max_vfs= 7 to any file in /etc/mo d pro be. d to make the

Virtual Functions persistent. For example:

# echo "options igb max_vfs=7" >>/etc/modprobe.d/igb.conf

110

6. In sp ect t h e n ew Virt u al Fu n ct io n s

Using the l spci command, list the newly added Virtual Functions attached to the Intel 82576 network device. (Alternatively, use g rep to search for Vi rtual Functi o n, to search for devices that support Virtual Functions.)

# lspci | grep 82576

0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection (rev 01)

0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit

Network Connection(rev 01)

0b:10.0 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.1 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.2 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.3 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.4 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.5 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.6 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:10.7 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.0 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.1 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.2 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.3 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.4 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

0b:11.5 Ethernet controller: Intel Corporation 82576 Virtual

Function (rev 01)

The identifier for the PCI device is found with the -n parameter of the l spci command. The

Physical Functions correspond to 0 b: 0 0 . 0 and 0 b: 0 0 . 1. All Virtual Functions have

Vi rtual Functi o n in the description.

7. Verif y d evices exist wit h virsh

The l i bvi rt service must recognize the device before adding a device to a virtual machine.

l i bvi rt uses a similar notation to the l spci output. All punctuation characters, ; and . , in

l spci output are changed to underscores (_).

Use the vi rsh no d ed ev-l i st command and the g rep command to filter the Intel 82576

⁠Chapt er 1 3. SR- IO V network device from the list of available host devices. 0b is the filter for the Intel 82576 network devices in this example. This may vary for your system and may result in additional devices.

# virsh nodedev-list | grep 0b pci_0000_0b_00_0 pci_0000_0b_00_1 pci_0000_0b_10_0 pci_0000_0b_10_1 pci_0000_0b_10_2 pci_0000_0b_10_3 pci_0000_0b_10_4 pci_0000_0b_10_5 pci_0000_0b_10_6 pci_0000_0b_11_7 pci_0000_0b_11_1 pci_0000_0b_11_2 pci_0000_0b_11_3 pci_0000_0b_11_4 pci_0000_0b_11_5

The serial numbers for the Virtual Functions and Physical Functions should be in the list.

8. G et d evice d et ails wit h virsh

The pci _0 0 0 0 _0 b_0 0 _0 is one of the Physical Functions and pci _0 0 0 0 _0 b_10 _0 is the first corresponding Virtual Function for that Physical Function. Use the vi rsh

no d ed ev-d umpxml command to get advanced output for both devices.

# virsh nodedev-dumpxml pci_0000_0b_00_0

</driver>

<product id='0x10c9'>82576 Gigabit Network

Connection</product>

<vendor id='0x8086'>Intel Corporation</vendor>

</capability>

</device>

# virsh nodedev-dumpxml pci_0000_0b_10_0

<name>igbvf</name>

</driver>

111

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

<product id='0x10ca'>82576 Virtual Function</product>

<vendor id='0x8086'>Intel Corporation</vendor>

</capability>

</device>

This example adds the Virtual Function pci _0 0 0 0 _0 b_10 _0 to the virtual machine in Step

9 . Note the bus, sl o t and functi o n parameters of the Virtual Function: these are required

for adding the device.

Copy these parameters into a temporary XML file, such as /tmp/new-i nterface. xml for example.

</source>

</interface>

112

⁠Chapt er 1 3. SR- IO V

Note

If you do not specify a MAC address, one will be automatically generated. The

<virtualport> element is only used when connecting to an 802.11Qbh hardware switch. The <vlan> element is new for Red Hat Enterprise Linux 6.4 and this will transparently put the guest's device on the VLAN tagged 42.

When the virtual machine starts, it should see a network device of the type provided by the physical adapter, with the configured MAC address. This MAC address will remain unchanged across host and guest reboots.

The following <interface> example shows the syntax for the optional <mac

address>, <virtualport>, and <vlan> elements. In practice, use either the <vlan> or <virtualport> element, not both simultaneously as shown in the example:

...

...

</source>

<vlan>

</vlan>

</virtualport>

</interface>

...

</devices>

Ad d t h e Virt u al Fu n ct io n t o t h e virt u al mach in e

Add the Virtual Function to the virtual machine using the following command with the temporary file created in the previous step. This attaches the new device immediately and saves it for subsequent guest restarts.

virsh attach-device MyGuest /tmp/new-i nterface. xml --config

Using the --co nfi g option ensures the new device is available after future guest restarts.

The virtual machine detects a new network interface card. This new card is the Virtual Function of the

SR-IOV device.

13.3. T roubleshoot ing SR-IOV

This section contains solutions for problems which may affect SR-IOV.

113

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Erro r st art in g t h e g u est

When starting a configured virtual machine, an error occurs as follows:

# virsh start test error: Failed to start domain test error: internal error unable to start guest: char device redirected to

/dev/pts/2 get_real_device: /sys/bus/pci/devices/0000:03:10.0/config:

Permission denied init_assigned_device: Error: Couldn't get real device (03:10.0)!

Failed to initialize assigned device host=03:10.0

This error is often caused by a device that is already assigned to another guest or to the host itself.

Erro r mig rat in g , savin g , o r d u mp in g t h e g u est

Attempts to migrate and dump the virtual machine cause an error similar to the following:

# virsh dump --crash 5 /tmp/vmcore error: Failed to core dump domain 5 to /tmp/vmcore error: internal error unable to execute QEMU command 'migrate':

An undefined error has occurred

Because device assignment uses hardware on the specific host where the virtual machine was started, guest migration and save are not supported when device assignment is in use.

Currently, the same limitation also applies to core-dumping a guest; this may change in the future.

114

⁠Chapt er 1 4 . KVM guest t iming management

Chapter 14. KVM guest timing management

Virtualization involves several intrinsic challenges for time keeping in guest virtual machines.

Interrupts cannot always be delivered simultaneously and instantaneously to all guest virtual machines, because interrupts in virtual machines are not true interrupts; they are injected into the guest virtual machine by the host machine. The host may be running another guest virtual machine, or a different process, meaning that the precise timing typically required by interrupts may not always be possible.

Guest virtual machines without accurate time keeping may experience issues with network applications and processes, as session validity, migration, and other network activities rely on timestamps to remain correct.

KVM avoids these issues by providing guest virtual machines with a para-virtualized clock (kvm-

clo ck). However, it is still vital to test timing before attempting activities that may be affected by time keeping inaccuracies.

Note

Red Hat Enterprise Linux 5.5 and newer, and Red Hat Enterprise Linux 6.0 and newer, use

kvm- clo ck as their default clock source. Running without kvm- clo ck requires special configuration, and is not recommended.

Important

The Network Time Protocol (NTP) daemon should be running on the host and the guest virtual machines. Enable the ntpd service:

# service ntpd start

Add the ntpd service to the default startup sequence:

# chkconfig ntpd on

The ntpd service will correct the effects of clock skew as long as the clock runs no more than

0.05% faster or slower than the reference time source. The ntp startup script adjusts the clock offset from the reference time by adjusting the system clock at startup time, if required.

14 .1. Const ant T ime St amp Count er (T SC)

Modern Intel and AMD CPUs provide a constant Time Stamp Counter (TSC). The count frequency of the constant TSC does not vary when the CPU core itself changes frequency, for example, to comply with a power saving policy. A CPU with a constant TSC frequency is necessary in order to use the

TSC as a clock source for KVM guests.

Your CPU has a constant Time Stamp Counter if the co nstant_tsc flag is present. To determine if your CPU has the co nstant_tsc flag run the following command:

$ cat /proc/cpuinfo | grep constant_tsc

115

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

If any output is given your CPU has the co nstant_tsc bit. If no output is given follow the instructions below.

14 .1.1. Configuring host s wit hout a const ant T ime St amp Count er

Systems without a constant TSC frequency cannot use the TSC as a clock source for virtual machines, and require additional configuration. Power management features interfere with accurate time keeping and must be disabled for guest virtual machines to accurately keep time with KVM.

Important

These instructions are for AMD revision F CPUs only.

If the CPU lacks the co nstant_tsc bit, disable all power management features ( BZ #513138 ). Each system has several timers it uses to keep time. The TSC is not stable on the host, which is sometimes caused by cpufreq changes, deep C state, or migration to a host with a faster TSC. Deep C sleep states can stop the TSC. To prevent the kernel using deep C states append

pro cesso r. max_cstate= 1 to the kernel boot options in the g rub. co nf file on the host: title Red Hat Enterprise Linux (2.6.32-330.x86_64)

root (hd0,0)

kernel /vmlinuz-2.6.32-330.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb quiet \

processor.max_cstate=1

Disable cpufreq (only necessary on hosts without the co nstant_tsc) by editing the

/etc/sysco nfi g /cpuspeed configuration file and change the MIN_SP EED and MAX_SP EED variables to the highest frequency available. Valid limits can be found in the

/sys/d evi ces/system/cpu/cpu*/cpufreq /scal i ng _avai l abl e_freq uenci es files.

14 .2. Required paramet ers for Red Hat Ent erprise Linux guest s

For certain Red Hat Enterprise Linux guest virtual machines, additional kernel parameters are required. These parameters can be set by appending them to the end of the /kernel line in the

/bo o t/g rub/g rub. co nf file of the guest virtual machine.

The table below lists versions of Red Hat Enterprise Linux and the parameters required on the specified systems.

T ab le 14 .1. Kern el p aramet er req u iremen t s

Red Hat En t erp rise Lin u x versio n

6.0 AMD64/Intel 64 with the para-virtualized clock

6.0 AMD64/Intel 64 without the para-virtualized clock

5.5 AMD64/Intel 64 with the para-virtualized clock

5.5 AMD64/Intel 64 without the para-virtualized clock

Ad d it io n al g u est kern el p aramet ers

Additional parameters are not required notsc lpj=n

116

Red Hat En t erp rise Lin u x versio n

5.5 x86 with the para-virtualized clock

5.5 x86 without the para-virtualized clock

5.4 AMD64/Intel 64

5.4 x86

5.3 AMD64/Intel 64

5.3 x86

4.8 AMD64/Intel 64

4.8 x86

3.9 AMD64/Intel 64

3.9 x86

⁠Chapt er 1 4 . KVM guest t iming management

Ad d it io n al g u est kern el p aramet ers

Additional parameters are not required clocksource=acpi_pm lpj=n notsc clocksource=acpi_pm notsc clocksource=acpi_pm notsc clock=pmtmr

Additional parameters are not required

Note

The lpj parameter requires a numeric value equal to the loops per jiffy value of the specific

CPU on which the guest virtual machine runs. If you do not know this value, do not set the lpj parameter.

Warning

The divider kernel parameter was previously recommended for Red Hat Enterprise Linux 4 and 5 guest virtual machines that did not have high responsiveness requirements, or exist on systems with high guest density. It is no longer recommended for use with guests running Red

Hat Enterprise Linux 4, or Red Hat Enterprise Linux 5 versions prior to version 5.8.

divider can improve throughput on Red Hat Enterprise Linux 5 versions equal to or later than 5.8 by lowering the frequency of timer interrupts. For example, if HZ= 10 0 0 , and divider is set to 10 (that is, d i vi d er= 10 ), the number of timer interrupts per period changes from the default value (1000) to 100 (the default value, 1000, divided by the divider value, 10).

BZ #698842 details a bug in the way that the divider parameter interacts with interrupt and tick recording. This bug is fixed as of Red Hat Enterprise Linux 5.8. However, the divider parameter can still cause kernel panic in guests using Red Hat Enterprise Linux 4, or Red Hat

Enterprise Linux 5 versions prior to version 5.8.

This parameter was not implemented in Red Hat Enterprise Linux 3, so this bug does not affect

Red Hat Enterprise Linux 3 guests.

Red Hat Enterprise Linux 6 does not have a fixed-frequency clock interrupt; it operates in

tickless mode and uses the timer dynamically as required. The divider parameter is therefore not useful for Red Hat Enterprise Linux 6, and Red Hat Enterprise Linux 6 guests are not affected by this bug.

14 .3. Using t he Real-T ime Clock wit h Windows Server 2008, Windows

Server 2008 R2, and Windows 7 guest s

117

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Windows uses both the Real-Time Clock (RTC) and the Time Stamp Counter (TSC). For Windows guest virtual machines the Real-Time Clock can be used instead of the TSC for all time sources, which resolves guest timing issues.

The bo o t. i ni file is no longer used as of Windows Server 2008 and newer. Windows Server 2008,

Windows Server 2008 R2, and Windows 7 do not use the TSC as a time source if the hypervi so r-

present bit is set. The Red Hat Enterprise Linux 6 KVM hypervisor enables this CPUID bit by default, so it is no longer necessary to use the Bo o t Co n f ig u rat io n Dat a Ed it o r (bcd ed i t. exe) to modify the Windows boot parameters.

Pro ced u re 14 .1. Usin g t h e Real- T ime Clo ck wit h Win d o ws Server 2008 R2 an d Win d o ws 7 g u est s

1. Open the Windows guest virtual machine.

2. Open the Accesso ries menu of the st art menu. Right click on the Co mman d Pro mp t application, select R un as Ad mi ni strato r.

3. Confirm the security exception, if prompted.

4. Set the boot manager to use the platform clock. This should instruct Windows to use the PM timer for the primary clock source. The system UUID ({default} in the example below) should be changed if the system UUID is different than the default boot device.

C:\Windows\system32>bcdedit /set {default} USEPLATFORMCLOCK on

The operation completed successfully

This fix should improve time keeping for Windows Server 2008 R2 and Windows 7 guests. Windows

2008 (non-R2) does not support the USEPLATFORMCLOCK parameter, but already uses the Real-Time

Clock by default.

14 .4 . St eal t ime account ing

Steal time is the amount of CPU time desired by a guest virtual machine that is not provided by the host. Steal time occurs when the host allocates these resources elsewhere: for example, to another guest.

Steal time is reported in the CPU time fields in /pro c/stat as st. It is automatically reported by utilities such as to p and vmstat, and cannot be switched off.

Large amounts of steal time indicate CPU contention, which can reduce guest performance. To relieve CPU contention, increase the guest's CPU priority or CPU quota, or run fewer guests on the host.

118

⁠Chapt er 1 5. Net work boot ing wit h libvirt

Chapter 15. Network booting with libvirt

Guest virtual machines can be booted with PXE enabled. PXE allows guest virtual machines to boot and load their configuration off the network itself. This section demonstrates some basic configuration steps to configure PXE guests with libvirt.

This section does not cover the creation of boot images or PXE servers. It is used to explain how to configure libvirt, in a private or bridged network, to boot a guest virtual machine with PXE booting enabled.

Warning

These procedures are provided only as an example. Ensure that you have sufficient backups before proceeding.

15.1. Preparing t he boot server

To perform the steps in this chapter you will need:

A PXE Server (DHCP and TFTP) - This can be a libvirt internal server, manually-configured dhcpd and tftpd, dnsmasq, a server configured by Cobbler, or some other server.

Boot images - for example, PXELINUX configured manually or by Cobbler.

15.1.1. Set t ing up a PXE boot server on a privat e libvirt net work

This example uses the default network. Perform the following steps:

Pro ced u re 15.1. Co n f ig u rin g t h e PXE b o o t server

1. Place the PXE boot images and configuration in /var/l i b/tftp.

2. Run the following commands:

# virsh net-destroy default

# virsh net-edit default

3. Edit the  element in the configuration file for the default network to include the appropriate address, network mask, DHCP address range, and boot file, where

BOOT_FILENAME represents the file name you are using to boot the guest virtual machine.

<dhcp>

</dhcp>

</ip>

4. Boot the guest using PXE (refer to Section 15.2, “ Booting a guest using PXE” ).

15.2. Boot ing a guest using PXE

119

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

15.2. Boot ing a guest using PXE

This section demonstrates how to boot a guest virtual machine with PXE.

15.2.1. Using bridged net working

Pro ced u re 15.2. Bo o t in g a g u est u sin g PXE an d b rid g ed n et wo rkin g

1. Ensure bridging is enabled such that the PXE boot server is available on the network.

2. Boot a guest virtual machine with PXE booting enabled. You can use the vi rt-i nstal l command to create a new virtual machine with PXE booting enabled, as shown in the following example command: virt-install --pxe --network bridge=breth0 --prompt

Alternatively, ensure that the guest network is configured to use your bridged network, and that the XML guest configuration file has a <bo o t d ev= ' netwo rk' /> element inside the

<o s> element, as shown in the following example:

<os>

</os>

</interface>

15.2.2. Using a privat e libvirt net work

Pro ced u re 15.3. Usin g a p rivat e lib virt n et wo rk

1. Configure PXE booting on libvirt as shown in Section 15.1.1, “ Setting up a PXE boot server on a private libvirt network” .

2. Boot a guest virtual machine using libvirt with PXE booting enabled. You can use the vi rt-

i nstal l command to create/install a new virtual machine using PXE: virt-install --pxe --network network=default --prompt

Alternatively, ensure that the guest network is configured to use your bridged network, and that the

XML guest configuration file has a <bo o t d ev= ' netwo rk' /> element inside the <o s> element, as shown in the following example:

<os>

120

⁠Chapt er 1 5. Net work boot ing wit h libvirt

</os>

Also ensure that the guest virtual machine is connected to the private network:

</interface>

121

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Appendix A. NetKVM Driver Parameters

After the NetKVM driver is installed, you can configure it to better suit your environment. The parameters listed in this section can be configured in the Windows D evi ce Manag er

(d evmg mt. msc).

Important

Modifying the driver's parameters causes Windows to re-load that driver. This interrupts existing network activity.

Pro ced u re A.1. Co n f ig u rin g Net KVM Paramet ers

1. O p en D evi ce Manag er

Click on the Start button. In the right-hand pane, right-click on C o mputer, and click

Manag e. If prompted, click C o nti nue on the User Acco unt C o ntro l window. This opens the C o mputer Manag ement window.

In the left-hand pane of the C o mputer Manag ement window, click D evi ce Manag er.

2. Lo cat e t h e co rrect d evice

In the central pane of the C o mputer Manag ement window, click on the + symbol beside

Netwo rk ad apters.

Under the list of R ed Hat Vi rtIO Ethernet Ad apter devices, double-click on NetKVM.

This opens the P ro perti es window for that device.

3. View d evice p aramet ers

In the P ro perti es window, click on the Ad vanced tab.

4. Mo d if y d evice p aramet ers

Click on the parameter you wish to modify to display the options for that parameter.

Modify the options as appropriate, then click on O K to save your changes.

A.1. Configurable paramet ers for Net KVM

Lo g g in g p aramet ers

Lo g g in g .En ab le

A Boolean value that determines whether logging is enabled. The default value is 1

(enabled).

Lo g g in g .Level

An integer that defines the logging level. As the integer increases, so does the verbosity of the log. The default value is 0 (errors only). 1-2 adds configuration messages. 3-4 adds packet flow information. 5-6 adds interrupt and DPC level trace information.

122

⁠Appendix A. Net KVM Driver Paramet ers

Important

High logging levels will slow down your guest virtual machine.

Lo g g in g .St at ist ics( sec)

An integer that defines whether log statistics are printed, and the time in seconds between each periodical statistics printout. The default value is 0 (no logging statistics).

In it ial p aramet ers

Assig n MAC

A string that defines the locally-administered MAC address for the para-virtualized NIC. This is not set by default.

In it .Co n n ect io n Rat e( Mb )

An integer that represents the connection rate in megabytes. The default value for Windows

2008 and later is 10 0 0 0 .

In it .Do 802.1PQ

A Boolean value that enables Priority/VLAN tag population and removal support. The default value is 1 (enabled).

In it .UseMerg ed Bu f f ers

A Boolean value that enables merge-able RX buffers. The default value is 1 (enabled).

In it .UsePu b lish Even t s

A Boolean value that enables published event use. The default value is 1 (enabled).

In it .MT USiz e

An integer that defines the maximum transmission unit (MTU). The default value is 150 0 .

Any value from 500 to 65500 is acceptable.

In it .In d irect T x

Controls whether indirect ring descriptors are in use. The default value is D i sabl e, which disables use of indirect ring descriptors. Other valid values are Enabl e, which enables indirect ring descriptor usage; and Enabl e*, which enables conditional use of indirect ring descriptors.

In it .MaxT xBu f f ers

An integer that represents the amount of TX ring descriptors that will be allocated. The default value is 10 24 . Valid values are: 16, 32, 64, 128, 256, 512, or 1024.

In it .MaxRxBu f f ers

An integer that represents the amount of RX ring descriptors that will be allocated. The default value is 256 . Valid values are: 16, 32, 64, 128, 256, 512, or 1024.

O f f lo ad .T x.Ch ecksu m

Specifies the TX checksum offloading mode.

123

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

In Red Hat Enterprise Linux 6.4 and onward, the valid values for this parameter are Al l

(the default), which enables IP, TCP and UDP checksum offloading for both IPv4 and IPv6;

T C P /UD P (v4 ,v6 ), which enables TCP and UDP checksum offloading for both IPv4 and

IPv6; T C P /UD P (v4 ), which enables TCP and UDP checksum offloading for IPv4 only; and T C P (v4 ), which enables only TCP checksum offloading for IPv4 only.

In Red Hat Enterprise Linux 6.3 and earlier, the valid values for this parameter are

T C P /UD P (the default value), which enables TCP and UDP checksum offload; T C P , which enables only TCP checksum offload; or D i sabl e, which disables TX checksum offload.

O f f lo ad .T x.LSO

A Boolean value that enables TX TCP Large Segment Offload (LSO). The default value is 1

(enabled).

O f f lo ad .Rx.Ch ecksu m

Specifies the RX checksum offloading mode.

In Red Hat Enterprise Linux 6.4 and onward, the valid values for this parameter are Al l

(the default), which enables IP, TCP and UDP checksum offloading for both IPv4 and IPv6;

T C P /UD P (v4 ,v6 ), which enables TCP and UDP checksum offloading for both IPv4 and

IPv6; T C P /UD P (v4 ), which enables TCP and UDP checksum offloading for IPv4 only; and T C P (v4 ), which enables only TCP checksum offloading for IPv4 only.

In Red Hat Enterprise Linux 6.3 and earlier, the valid values are D i sabl e (the default), which disables RX checksum offloading; Al l , which enables TCP, UDP, and IP checksum offloading; T C P /UD P , which enables TCP and UDP checksum offloading; and T C P , which enables only TCP checksum offloading.

T est an d d eb u g p aramet ers

Important

Test and debug parameters should only be used for testing or debugging; they should not be used in production.

124

T est O n ly.DelayCo n n ect ( ms)

The period for which to delay connection upon startup, in milliseconds. The default value is

0 .

T est O n ly.DPCCh eckin g

Sets the DPC checking mode. 0 (the default) disables DPC checking. 1 enables DPC checking; each hang test verifies DPC activity and acts as if the DPC was spawned. 2 clears the device interrupt status and is otherwise identical to 1.

T est O n ly.Scat t er- G at h er

A Boolean value that determines whether scatter-gather functionality is enabled. The default value is 1 (enabled). Setting this value to 0 disables scatter-gather functionality and all dependent capabilities.

T est O n ly.In t erru p t Reco very

A Boolean value that determines whether interrupt recovery is enabled. The default value is

1 (enabled).

⁠Appendix A. Net KVM Driver Paramet ers

1 (enabled).

T est O n ly.Packet Filt er

A Boolean value that determines whether packet filtering is enabled. The default value is 1

(enabled).

T est O n ly.Bat ch Receive

A Boolean value that determines whether packets are received in batches, or singularly.

The default value is 1, which enables batched packet receipt.

T est O n ly.Pro miscu o u s

A Boolean value that determines whether promiscuous mode is enabled. The default value is 0 (disabled).

T est O n ly.An alyz eIPPacket s

A Boolean value that determines whether the checksum fields of outgoing IP packets are tested and verified for debugging purposes. The default value is 0 (no checking).

T est O n ly.RXT h ro t t le

An integer that determines the number of receive packets handled in a single DPC. The default value is 10 0 0 .

T est O n ly.UseSwT xCh ecksu m

A Boolean value that determines whether hardware checksumming is enabled. The default value is 0 (disabled).

125

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Appendix B. Common libvirt errors and troubleshooting

This appendix documents common lib virt -related problems and errors along with instructions for dealing with them.

Locate the error on the table below and follow the corresponding link under So l uti o n for detailed troubleshooting information.

T ab le B.1. Co mmo n lib virt erro rs

Erro r l i bvi rtd fai l ed to start

C anno t read C A certi fi cate

Fai l ed to co nnect so cket . . . :

P ermi ssi o n d eni ed

Other connectivity errors

Descrip t io n o f p ro b lem

The lib virt daemon failed to start. However, there is no information about this error in

/var/l o g /messag es.

This is one of several errors that occur when the URI fails to connect to the hypervisor.

These are other errors that occur when the URI fails to connect to the hypervisor.

The guest virtual machine cannot be started because the host and guest processors are different.

So lu t io n

Section B.1, “ lib virt d failed to start”

Section B.2, “ The URI failed to connect to the hypervisor”

Internal erro r g uest

C P U i s no t co mpati bl e wi th ho st C P U

Fai l ed to create d o mai n fro m vm. xml erro r: mo ni to r so cket d i d no t sho w up. : C o nnecti o n refused

Internal erro r canno t fi nd character d evi ce

(nul l )

The guest virtual machine (or domain) starting fails and returns this error or similar.

This error can occur when attempting to connect a guest's console. It reports that there is no serial console configured for the guest virtual machine.

No bo o t d evi ce

T he vi rtual netwo rk

"default" has no t been started

After building a guest virtual machine from an existing disk image, the guest booting stalls.

However, the guest can start successfully using the Q EMU command directly.

If the default network (or other locally-created network) is unable to start, any virtual machine configured to use that network for its connectivity will also fail to start.

Section B.3, “ The guest virtual machine cannot be started: i nternal erro r g uest

C P U i s no t co mpati bl e

wi th ho st C P U”

Section B.4, “ Guest starting fails with error: mo ni to r

so cket d i d no t sho w up”

Section B.5, “ Internal erro r canno t fi nd character

d evi ce (nul l )”

Section B.6, “ Guest virtual machine booting stalls with error: No bo o t d evi ce”

Section B.7, “ Virtual network

default has not been started”

126

⁠Appendix B. Common libvirt errors and t roubleshoot ing

Erro r

PXE boot (or DHCP) on guest failed

Guest can reach outside network, but cannot reach host when using macvtap interface

Descrip t io n o f p ro b lem

A guest virtual machine starts successfully, but is unable to acquire an IP address from

DHCP, boot using the PXE protocol, or both. This is often a result of a long forward delay time set for the bridge, or when the iptables package and kernel do not support checksum mangling rules.

A guest can communicate with other guests, but cannot connect to the host machine after being configured to use a macvtap (or type='direct') network interface.

This is actually not an error — it is the defined behavior of macvtap.

So lu t io n

Section B.8, “ PXE boot (or

DHCP) on guest failed”

Section B.9, “ Guest can reach outside network, but cannot reach host when using macvtap interface”

C o ul d no t ad d rul e to fi xup D HC P respo nse checksums o n netwo rk

'default'

Unabl e to ad d bri d g e br0 po rt vnet0 : No such d evi ce

Warni ng : co ul d no t o pen /d ev/net/tun: no vi rtual netwo rk emul ati o n q emu-kvm: netd ev tap,scri pt= /etc/myq emu-i fup,i d = ho stnet0 :

D evi ce ' tap' co ul d no t be i ni ti al i zed

Unabl e to reso l ve ad d ress name_of_host servi ce ' 4 9 155' : Name o r servi ce no t kno wn

Unabl e to al l o w access fo r d i sk path

/var/l i b/l i bvi rt/i mag es

/q emu. i mg : No such fi l e o r d i recto ry

This warning message is almost always harmless, but is often mistakenly seen as evidence of a problem.

This error message or the similar Fai l ed to ad d tap i nterface to bri d g e

'br0': No such d evi ce reveal that the bridge device specified in the guest's (or domain's)  definition does not exist.

The guest virtual machine does not start after configuring a

type= ' ethernet' (or 'generic ethernet') interface in the host system. This error or similar appears either in

l i bvi rtd . l o g ,

/var/l o g /l i bvi rt/q emu/n

ame_of_guest. l o g , or in both.

Q EMU guest migration fails and this error message appears with an unfamiliar hostname.

A guest virtual machine cannot be migrated because lib virt cannot access the disk image(s).

Section B.10, “ Could not add rule to fixup DHCP response checksums on network 'default'”

Section B.11, “ Unable to add bridge br0 port vnet0: No such device”

Section B.12, “ Guest is unable to start with error: warni ng : co ul d no t o pen

/d ev/net/tun”

Section B.13, “ Migration fails with Erro r: unabl e to

reso l ve ad d ress”

Section B.14, “ Migration fails with Unabl e to al l o w access fo r d i sk path:

No such fi l e o r

d i recto ry”

127

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Erro r Descrip t io n o f p ro b lem

No guest virtual machines are present when lib virt d is started

Unabl e to co nnect to server at ' ho st: 16 50 9 ' :

C o nnecti o n refused . . . erro r: fai l ed to co nnect to the hypervi so r

Common XML errors

The lib virt daemon is successfully started, but no guest virtual machines appear to be present when running

vi rsh l i st --al l .

While lib virt d should listen on

TCP ports for connections, the connection to the hypervisor fails.

lib virt uses XML documents to store structured data. Several common errors occur with XML documents when they are passed to lib virt through the

API. This entry provides instructions for editing guest

XML definitions, and details common errors in XML syntax and configuration.

So lu t io n

Section B.15, “ No guest virtual machines are present when

lib virt d is started”

Section B.16, “ Unable to connect to server at

'host:16509': Connection refused ... error: failed to connect to the hypervisor”

Section B.17, “ Common XML errors”

B.1. libvirt d failed t o st art

Symp t o m

The lib virt daemon does not start automatically. Starting the lib virt daemon manually fails as well:

# /etc/init.d/libvirtd start

* Caching service dependencies ...

[ ok ]

* Starting libvirtd ...

/usr/sbin/libvirtd: error: Unable to initialize network sockets.

Check /var/log/messages or run without --daemon for more info.

* start-stop-daemon: failed to start `/usr/sbin/libvirtd'

[ !! ]

* ERROR: libvirtd failed to start

Moreover, there is not ' mo re i nfo ' about this error in /var/l o g /messag es.

In vest ig at io n

Change lib virt ' s logging in /etc/l i bvi rt/l i bvi rtd . co nf by uncommenting the line below. To uncomment the line, open the /etc/l i bvi rt/l i bvi rtd . co nf file in a text editor, remove the hash (or #) symbol from the beginning of the following line, and save the change: log_outputs="3:syslog:libvirtd"

128

⁠Appendix B. Common libvirt errors and t roubleshoot ing

Note

This line is commented out by default to prevent lib virt from producing excessive log messages. After diagnosing the problem, it is recommended to comment this line again in the /etc/l i bvi rt/l i bvi rtd . co nf file.

Restart lib virt to determine if this has solved the problem.

If l i bvi rtd still does not start successfully, an error similar to the following will be shown in the /var/l o g /messag es file:

Feb 6 17:22:09 bart libvirtd: 17576: info : libvirt version:

0.9.9

Feb 6 17:22:09 bart libvirtd: 17576: error : virNetTLSContextCheckCertFile:92: Cannot read CA certificate

'/etc/pki/CA/cacert.pem': No such file or directory

Feb 6 17:22:09 bart /etc/init.d/libvirtd[17573]: start-stopdaemon: failed to start `/usr/sbin/libvirtd'

Feb 6 17:22:09 bart /etc/init.d/libvirtd[17565]: ERROR: libvirtd failed to start

The lib virt d man page shows that the missing cacert. pem file is used as TLS authority when lib virt is run in Li sten fo r T C P /IP co nnecti o ns mode. This means the --

listen parameter is being passed.

So lu t io n

Configure the lib virt daemon's settings with one of the following methods:

Install a CA certificate.

Note

For more information on CA certificates and configuring system authentication, refer to the Configuring Authentication chapter in the Red Hat Enterprise Linux 6

Deployment Guide.

Do not use TLS; use bare TCP instead. In /etc/l i bvi rt/l i bvi rtd . co nf set

l i sten_tl s = 0 and l i sten_tcp = 1. The default values are l i sten_tl s = 1 and l i sten_tcp = 0 .

Do not pass the --listen parameter. In /etc/sysco nfi g /l i bvi rtd . co nf change the LIBVIRTD_ARGS variable.

B.2. T he URI failed t o connect t o t he hypervisor

Several different errors can occur when connecting to the server (for example, when running vi rsh).

B.2.1. Cannot read CA cert ificat e

Symp t o m

129

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

When running a command, the following error (or similar) appears:

$ virsh -c name_of_uri list error: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory error: failed to connect to the hypervisor

In vest ig at io n

The error message is misleading about the actual cause. This error can be caused by a variety of factors, such as an incorrectly specified URI, or a connection that is not configured.

So lu t io n

In co rrect ly sp ecif ied URI

When specifying qemu://system or qemu://session as a connection URI,

vi rsh attempts to connect to hostnames system or session respectively. This is because vi rsh recognizes the text after the second forward slash as the host.

Use three forward slashes to connect to the local host. For example, specifying

qemu:///system instructs vi rsh connect to the system instance of lib virt d on the local host.

When a hostname is specified, the Q EMU transport defaults to T LS. This results in certificates.

Co n n ect io n is n o t co n f ig u red

The URI is correct (for example, qemu[+tls]://server/system) but the certificates are not set up properly on your machine. For information on configuring TLS, see Setting up lib virt for TLS available from the lib virt website.

B.2.2. Failed t o connect socket ... : Permission denied

Symp t o m

When running a vi rsh command, the following error (or similar) appears:

$ virsh -c qemu:///system list error: Failed to connect socket to '/var/run/libvirt/libvirtsock': Permission denied error: failed to connect to the hypervisor

In vest ig at io n

Without any hostname specified, the connection to Q EMU uses UNIX sockets by default. If there is no error running this command as root, the UNIX socket options in

/etc/l i bvi rt/l i bvi rtd . co nf are likely misconfigured.

So lu t io n

To connect as a non-root user using UNIX sockets, configure the following options in

/etc/l i bvi rt/l i bvi rtd . co nf:

130

⁠Appendix B. Common libvirt errors and t roubleshoot ing unix_sock_group = <g ro up> unix_sock_ro_perms = <perms> unix_sock_rw_perms = <perms>

Note

The user running vi rsh must be a member of the g ro up specified in the

uni x_so ck_g ro up option.

B.2.3. Ot her connect ivit y errors

Un ab le t o co n n ect t o server at server: po rt: Co n n ect io n ref u sed

The daemon is not running on the server or is configured not to listen, using configuration option l i sten_tcp or l i sten_tl s.

En d o f f ile wh ile read in g d at a: n c: u sin g st ream so cket : In p u t /o u t p u t erro r

If you specified ssh transport, the daemon is likely not running on the server. Solve this error by verifying that the daemon is running on the server.

B.3. T he guest virt ual machine cannot be st art ed:

i nternal erro r g uest

C P U i s no t co mpati bl e wi th ho st C P U

Symp t o m

Running on an Intel Core i7 processor (which virt - man ag er refers to as Nehal em, or the older Core 2 Duo, referred to as P enryn), a KVM guest (or domain) is created using virt -

man ag er. After installation, the guest's processor is changed to match the host's CPU. The guest is then unable to start and reports this error:

2012-02-06 17:49:15.985+0000: 20757: error : qemuBuildCpuArgStr:3565 : internal error guest CPU is not compatible with host CPU

Additionally, clicking C o py ho st C P U co nfi g urati o n in virt - man ag er shows

Pentium III instead of Nehal em or P enryn.

In vest ig at io n

The /usr/share/l i bvi rt/cpu_map. xml file lists the flags that define each CPU model.

The Nehal em and P enryn definitions contain this:

As a result, the NX (or No eXecute) flag needs to be presented to identify the CPU as

Nehal em or P enryn. However, in /pro c/cpui nfo , this flag is missing.

So lu t io n

131

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

Nearly all new BIOSes allow enabling or disabling of the No eXecute bit. However, if disabled, some CPUs do not report this flag and thus lib virt detects a different CPU.

Enabling this functionality instructs lib virt to report the correct CPU. Refer to your hardware documentation for further instructions on this subject.

B.4 . Guest st art ing fails wit h error:

mo ni to r so cket d i d no t sho w up

Symp t o m

The guest virtual machine (or domain) starting fails with this error (or similar):

# virsh -c qemu:///system create name_of_guest.xml error: Failed to create domain from name_of_guest.xml error: monitor socket did not show up.: Connection refused

In vest ig at io n

This error message shows:

1. lib virt is working;

2. The Q EMU process failed to start up; and

3. lib virt quits when trying to connect Q EMU or the QEMU agent monitor socket.

To understand the error details, examine the guest log:

# cat /var/log/libvirt/qemu/name_of_guest.log

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none

/usr/bin/qemu-kvm -S -M pc -enable-kvm -m 768 -smp

1,sockets=1,cores=1,threads=1 -name name_of_guest -uuid ebfaadbee908-ba92-fdb8-3fa2db557a42 -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/name_of_guest.monito

r,server,nowait -mon chardev=monitor,mode=readline -no-reboot boot c -kernel /var/lib/libvirt/boot/vmlinuz -initrd

/var/lib/libvirt/boot/initrd.img -append method=http://www.example.com/pub/product/release/version/x86_64/ os/ -drive file=/var/lib/libvirt/images/name_of_guest.img,if=none,id=driveide0-0-0,boot=on -device ide-drive,bus=ide.0,unit=0,drive=driveide0-0-0,id=ide0-0-0 -device virtio-netpci,vlan=0,id=net0,mac=52:40:00:f4:f1:0a,bus=pci.0,addr=0x4 -net tap,fd=42,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:0 -k en-gb -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0, addr=0x3 char device redirected to /dev/pts/1 qemu: could not load kernel '/var/lib/libvirt/boot/vmlinuz':

Permission denied

So lu t io n

The guest log contains the details needed to fix the error.

If a host is shut down while the guest is still running a lib virt version prior to 0.9.5, the libvirt-guest's init script attempts to perform a managed save of the guest. If the managed

132

⁠Appendix B. Common libvirt errors and t roubleshoot ing save was incomplete (for example, due to loss of power before the managed save image was flushed to disk), the save image is corrupted and will not be loaded by Q EMU. The older version of lib virt does not recognize the corruption, making the problem perpetual. In this case, the guest log will show an attempt to use -i nco mi ng as one of its arguments, meaning that lib virt is trying to start Q EMU by migrating in the saved state file.

This problem can be fixed by running vi rsh manag ed save-remo ve name_of_guest to remove the corrupted managed save image. Newer versions of lib virt take steps to avoid the corruption in the first place, as well as adding vi rsh start --fo rce-bo o t

name_of_guest to bypass any managed save image.

B.5.

Internal erro r canno t fi nd character d evi ce (nul l )

Symp t o m

This error message appears when attempting to connect to a guest virtual machine's console:

# virsh console test2 Connected to domain test2 Escape character is ^] error: internal error cannot find character device (null)

In vest ig at io n

This error message shows that there is no serial console configured for the guest virtual machine.

So lu t io n

Set up a serial console in the guest's XML file.

Pro ced u re B.1. Set t in g u p a serial co n so le in t h e g u est ' s XML

1. Add the following XML to the guest virtual machine's XML using virsh ed it :

</serial>

</console>

2. Set up the console in the guest kernel command line.

To do this, either log in to the guest virtual machine to edit the

/bo o t/g rub/g rub. co nf file directly, or use the virt - ed it command line tool. Add the following to the guest kernel command line: console=ttyS0,115200

3. Run the followings command:

# virsh start vm & & virsh console vm

B.6. Guest virt ual machine boot ing st alls wit h error:

No bo o t d evi ce

133

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

134

Symp t o m

After building a guest virtual machine from an existing disk image, the guest booting stalls with the error message No bo o t d evi ce. However, the guest virtual machine can start successfully using the Q EMU command directly.

In vest ig at io n

The disk's bus type is not specified in the command for importing the existing disk image:

# virt-install \

--connect qemu:///system \

--ram 2048 -n rhel_64 \

--os-type=linux --os-variant=rhel5 \

--disk path=/root/RHEL-Server-5.8-64virtio.qcow2,device=disk,format=qcow2 \

--vcpus=2 --graphics spice --noautoconsole --import

However, the command line used to boot up the guest virtual machine using Q EMU directly shows that it uses vi rti o for its bus type:

# ps -ef | grep qemu

/usr/libexec/qemu-kvm -monitor stdio -drive file=/root/RHEL-

Server-5.8-32virtio.qcow2,index=0,i f= vi rti o ,media=disk,cache=none,format=qcow2

-net nic,vlan=0,model=rtl8139,macaddr=00:30:91:aa:04:74 -net tap,vlan=0,script=/etc/qemu-ifup,downscript=no -m 2048 -smp

2,cores=1,threads=1,sockets=2 -cpu qemu64,+sse2 -soundhw ac97 rtc-td-hack -M rhel5.6.0 -usbdevice tablet -vnc :10 -boot c -nokvm-pit-reinjection

Note the bus= in the guest's XML generated by lib virt for the imported guest:

<os>

</os>

<acpi/>

<apic/>

<pae/>

</features>

</clock>

<on_poweroff>destroy</on_poweroff>

<on_reboot>restart</on_reboot>

<on_crash>restart</on_crash>

⁠Appendix B. Common libvirt errors and t roubleshoot ing

<emulator>/usr/libexec/qemu-kvm</emulator>

</emphasis>

</disk>

</interface>

</serial>

</console>

<video>

</video>

</devices>

</domain>

The bus type for the disk is set as ide, which is the default value set by lib virt . This is the incorrect bus type, and has caused the unsuccessful boot for the imported guest.

So lu t io n

Pro ced u re B.2. Co rrect in g t h e d isk b u s t yp e

1. Undefine the imported guest, then re-import it with bus=virtio and the following:

# virsh destroy rhel_64

# virsh undefine rhel_64

# virt-install \

--connect qemu:///system \

--ram 1024 -n rhel_64 -r 2048 \

--os-type=linux --os-variant=rhel5 \

--disk path=/root/RHEL-Server-5.8-64virtio.qcow2,device=disk,bus= vi rti o ,format=qcow2 \

--vcpus=2 --graphics spice --noautoconsole --import

2. Edit the imported guest's XML using vi rsh ed i t and correct the disk bus type.

**B.7. Virt ual net work default has not been st art ed**

Symp t o m

Normally, the configuration for a virtual network named default is installed as part of the

libvirt package, and is configured to autostart when lib virt d is started.

135

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

If the default network (or any other locally-created network) is unable to start, any virtual machine configured to use that network for its connectivity will also fail to start, resulting in this error message:

Virtual network default has not been started

In vest ig at io n

One of the most common reasons for a lib virt virtual network's failure to start is that the dnsmasq instance required to serve DHCP and DNS requests from clients on that network has failed to start.

To determine if this is the cause, run vi rsh net-start d efaul t from a root shell to start the default virtual network.

If this action does not successfully start the virtual network, open

/var/l o g /l i bvi rt/l i bvi rtd . l o g to view the complete error log message.

If a message similar to the following appears, the problem is likely a systemwide dnsmasq instance that is already listening on lib virt 's bridge, and is preventing lib virt 's own dnsmasq instance from doing so. The most important parts to note in the error message are

d nsmasq and exi t status 2:

Could not start virtual network default: internal error

Child process (/usr/sbin/dnsmasq --strict-order --bind-interfaces

--pid-file=/var/run/libvirt/network/default.pid --conf-file=

--except-interface lo --listen-address 192.168.122.1

--dhcp-range 192.168.122.2,192.168.122.254

--dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases

--dhcp-lease-max=253 --dhcp-no-override) status unexpected: exit status 2

So lu t io n

If the machine is not using dnsmasq to serve DHCP for the physical network, disable dnsmasq completely.

If it is necessary to run dnsmasq to serve DHCP for the physical network, edit the

/etc/d nsmasq . co nf file. Add or uncomment the first line, as well as one of the two lines following that line. Do not add or uncomment all three lines: bind-interfaces interface=name_of_physical_interface listen-address=chosen_IP_address

After making this change and saving the file, restart the systemwide dnsmasq service.

Next, start the default network with the vi rsh net-start d efaul t command.

Start the virtual machines.

B.8. PXE boot (or DHCP) on guest failed

136

Symp t o m

A guest virtual machine starts successfully, but is then either unable to acquire an IP address from DHCP or boot using the PXE protocol, or both. There are two common causes

⁠Appendix B. Common libvirt errors and t roubleshoot ing address from DHCP or boot using the PXE protocol, or both. There are two common causes of this error: having a long forward delay time set for the bridge, and when the iptables package and kernel do not support checksum mangling rules.

Lo n g f o rward d elay t ime o n b rid g e

In vest ig at io n

This is the most common cause of this error. If the guest network interface is connecting to a bridge device that has STP (Spanning Tree

Protocol) enabled, as well as a long forward delay set, the bridge will not forward network packets from the guest virtual machine onto the bridge until at least that number of forward delay seconds have elapsed since the guest connected to the bridge. This delay allows the bridge time to watch traffic from the interface and determine the MAC addresses behind it, and prevent forwarding loops in the network topology.

If the forward delay is longer than the timeout of the guest's PXE or

DHCP client, then the client's operation will fail, and the guest will either fail to boot (in the case of PXE) or fail to acquire an IP address (in the case of DHCP).

So lu t io n

If this is the case, change the forward delay on the bridge to 0, or disable STP on the bridge.

Note

This solution applies only if the bridge is not used to connect multiple networks, but just to connect multiple endpoints to a single network (the most common use case for bridges used by

lib virt ).

If the guest has interfaces connecting to a lib virt -managed virtual network, edit the definition for the network, and restart it. For example, edit the default network with the following command:

# virsh net-edit default

Add the following attributes to the <bri d g e> element:

<name_of_bridge='virbr0' d el ay= ' 0 ' stp= ' o n' />

Note

d el ay= ' 0 ' and stp= ' o n' are the default settings for virtual networks, so this step is only necessary if the configuration has been modified from the default.

If the guest interface is connected to a host bridge that was configured outside of lib virt , change the delay setting.

Add or edit the following lines in the /etc/sysco nfi g /netwo rk-

137

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

scri pts/i fcfg -name_of_bridge file to turn STP on with a 0 second delay:

STP=on

DELAY=0

After changing the configuration file, restart the bridge device:

/sbin/ifdown name_of_bridge

/sbin/ifup name_of_bridge

Note

If name_of_bridge is not the root bridge in the network, that bridge's delay will eventually reset to the delay time configured for the root bridge. In this case, the only solution is to disable STP completely on name_of_bridge.

T h e iptables p ackag e an d kern el d o n o t su p p o rt ch ecksu m man g lin g ru les

In vest ig at io n

This message is only a problem if all four of the following conditions are true:

The guest is using virt io network devices.

If so, the configuration file will contain mo d el type= ' vi rti o '

The host has the vho st-net module loaded.

This is true if l s /d ev/vho st-net does not return an empty result.

The guest is attempting to get an IP address from a DHCP server that is running directly on the host.

The iptables version on the host is older than 1.4.10.

iptables 1.4.10 was the first version to add the l i bxt_C HEC KSUM extension. This is the case if the following message appears in the

lib virt d logs: warning: Could not add rule to fixup DHCP response checksums on network default warning: May need to update iptables package and kernel to support CHECKSUM rule.

Important

Unless all of the other three conditions in this list are also true, the above warning message can be disregarded, and is not an indicator of any other problems.

138

⁠Appendix B. Common libvirt errors and t roubleshoot ing

When these conditions occur, UDP packets sent from the host to the guest have uncomputed checksums. This makes the host's UDP packets seem invalid to the guest's network stack.

So lu t io n

To solve this problem, invalidate any of the four points above. The best solution is to update the host iptables and kernel to iptables-1.4.10 or later where possible. Otherwise, the most specific fix is to disable the vho st-

net driver for this particular guest. To do this, edit the guest configuration with this command: virsh edit name_of_guest

Change or add a <d ri ver> line to the  section:

...

</interface>

Save the changes, shut down the guest, and then restart it.

If this problem is still not resolved, the issue may be due to a conflict between f irewalld and the default lib virt network.

To fix this, stop f irewalld with the servi ce fi rewal l d sto p command, then restart lib virt with the servi ce l i bvi rtd restart command.

B.9. Guest can reach out side net work, but cannot reach host when using macvt ap int erface

Symp t o m

A guest virtual machine can communicate with other guests, but cannot connect to the host machine after being configured to use a macvtap (also known as type='direct') network interface.

In vest ig at io n

Even when not connecting to a Virtual Ethernet Port Aggregator (VEPA) or VN-Link capable switch, macvtap interfaces can be useful. Setting the mode of such an interface to bri d g e allows the guest to be directly connected to the physical network in a very simple manner without the setup issues (or NetworkManager incompatibility) that can accompany the use of a traditional host bridge device.

However, when a guest virtual machine is configured to use a type='direct' network interface such as macvtap, despite having the ability to communicate with other guests and other external hosts on the network, the guest cannot communicate with its own host.

This situation is actually not an error — it is the defined behavior of macvtap. Due to the way in which the host's physical Ethernet is attached to the macvtap bridge, traffic into that bridge from the guests that is forwarded to the physical interface cannot be bounced back up to the host's IP stack. Additionally, traffic from the host's IP stack that is sent to the

139

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide physical interface cannot be bounced back up to the macvtap bridge for forwarding to the guests.

So lu t io n

Use lib virt to create an isolated network, and create a second interface for each guest virtual machine that is connected to this network. The host and guests can then directly communicate over this isolated network, while also maintaining compatibility with

NetworkManager.

Pro ced u re B.3. Creat in g an iso lat ed n et wo rk wit h lib virt

1. Add and save the following XML in the /tmp/i so l ated . xml file. If the

192.168.254.0/24 network is already in use elsewhere on your network, you can choose a different network.

<name>isolated</name>

<dhcp>

</dhcp>

</ip>

</network>

2. Create the network with this command: vi rsh net-d efi ne

/tmp/i so l ated . xml

3. Set the network to autostart with the vi rsh net-auto start i so l ated command.

4. Start the network with the vi rsh net-start i so l ated command.

5. Using vi rsh ed i t name_of_guest, edit the configuration of each guest that uses macvtap for its network connection and add a new <interface> in the

<devices> section similar to the following (note the <model type='virtio'/> line is optional to include):

</interface>

6. Shut down, then restart each of these guests.

The guests are now able to reach the host at the address 192.168.254.1, and the host will be able to reach the guests at the IP address they acquired from DHCP (alternatively, you can manually configure the IP addresses for the guests). Since this new network is isolated to only the host and guests, all other communication from the guests will use the macvtap interface.

B.10. Could not add rule t o fixup DHCP response checksums on net work 'default'

14 0

Symp t o m

This message appears:

⁠Appendix B. Common libvirt errors and t roubleshoot ing

This message appears:

Could not add rule to fixup DHCP response checksums on network

'default'

In vest ig at io n

Although this message appears to be evidence of an error, it is almost always harmless.

So lu t io n

Unless the problem you are experiencing is that the guest virtual machines are unable to acquire IP addresses through DHCP, this message can be ignored.

If this is the case, refer to

Section B.8, “ PXE boot (or DHCP) on guest failed”

for further details on this situation.

B.11. Unable t o add bridge br0 port vnet 0: No such device

Symp t o m

The following error message appears:

Unable to add bridge name_of_bridge port vnet0: No such device

For example, if the bridge name is br0, the error message will appear as:

Unable to add bridge br0 port vnet0: No such device

In lib virt versions 0.9.6 and earlier, the same error appears as:

Failed to add tap interface to bridge name_of_bridge: No such device

Or for example, if the bridge is named br0:

Failed to add tap interface to bridge 'br0': No such device

In vest ig at io n

Both error messages reveal that the bridge device specified in the guest's (or domain's)

 definition does not exist.

To verify the bridge device listed in the error message does not exist, use the shell command i fco nfi g br0.

A message similar to this confirms the host has no bridge by that name:

br0: error fetching interface information: Device not found

If this is the case, continue to the solution.

However, if the resulting message is similar to the following, the issue exists elsewhere:

14 1

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

br0 Link encap:Ethernet HWaddr 00:00:5A:11:70:48

inet addr:10.22.1.5 Bcast:10.255.255.255

Mask:255.0.0.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:249841 errors:0 dropped:0 overruns:0 frame:0

TX packets:281948 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:106327234 (101.4 MiB) TX bytes:21182634 (20.2 MiB)

So lu t io n

Ed it t h e exist in g b rid g e o r creat e a n ew b rid g e wit h vi rsh

Use vi rsh to either edit the settings of an existing bridge or network, or to add the bridge device to the host system configuration.

Ed it t h e exist in g b rid g e set t in g s u sin g vi rsh

Use vi rsh ed i t name_of_guest to change the  definition to use a bridge or network that already exists.

For example, change type= ' bri d g e' to type= ' netwo rk' , and

<so urce bri d g e= ' br0 ' /> to <so urce netwo rk= ' d efaul t' />.

Creat e a h o st b rid g e u sin g vi rsh

For lib virt version 0.9.8 and later, a bridge device can be created with the vi rsh i face-bri d g e command. This will create a bridge device

br0 with eth0 , the physical network interface which is set as part of a bridge, attached: virsh iface-bridge eth0 br0

Optional: If desired, remove this bridge and restore the original eth0 configuration with this command: virsh iface-unbridge br0

Creat e a h o st b rid g e man u ally

For older versions of lib virt , it is possible to manually create a bridge device on the host. Refer to

Section 11.3, “ Bridged networking with libvirt”

for instructions.

B.12. Guest is unable t o st art wit h error:

warni ng : co ul d no t o pen

/d ev/net/tun

Symp t o m

The guest virtual machine does not start after configuring a type= ' ethernet' (also known as 'generic ethernet') interface in the host system. An error appears either in

l i bvi rtd . l o g , /var/l o g /l i bvi rt/q emu/name_of_guest. l o g , or in both, similar to the below message:

14 2

⁠Appendix B. Common libvirt errors and t roubleshoot ing warning: could not open /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/my-qemuifup,id=hostnet0: Device 'tap' could not be initialized

In vest ig at io n

Use of the generic ethernet interface type () is discouraged, because using it requires lowering the level of host protection against potential security flaws in Q EMU and its guests. However, it is sometimes necessary to use this type of interface to take advantage of some other facility that is not yet supported directly in lib virt . For example, o p en vswit ch was not supported in lib virt until libvirt-

0.9.11, so in older versions of lib virt ,  was the only way to connect a guest to an o p en vswit ch bridge.

However, if you configure a  interface without making any other changes to the host system, the guest virtual machine will not start successfully.

The reason for this failure is that for this type of interface, a script called by Q EMU needs to manipulate the tap device. However, with type= ' ethernet' configured, in an attempt to lock down Q EMU, lib virt and SELinux have put in place several checks to prevent this.

(Normally, lib virt performs all of the tap device creation and manipulation, and passes an open file descriptor for the tap device to Q EMU.)

So lu t io n

Reconfigure the host system to be compatible with the generic ethernet interface.

Pro ced u re B.4 . Reco n f ig u rin g t h e h o st syst em t o u se t h e g en eric et h ern et in t erf ace

1. Set SELinux to permissive by configuring SELINUX= permi ssi ve in

/etc/sel i nux/co nfi g :

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - No SELinux policy is loaded.

SELINUX=permissive

# SELINUXTYPE= can take one of these two values:

# targeted - Targeted processes are protected,

# mls - Multi Level Security protection.

SELINUXTYPE=targeted

2. From a root shell, run the command setenfo rce permi ssi ve.

3. In /etc/l i bvi rt/q emu. co nf add or edit the following lines: clear_emulator_capabilities = 0 user = "root" group = "root"

14 3

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide cgroup_device_acl = [

"/dev/null", "/dev/full", "/dev/zero",

"/dev/random", "/dev/urandom",

"/dev/ptmx", "/dev/kvm", "/dev/kqemu",

"/dev/rtc", "/dev/hpet", "/dev/net/tun",

4. Restart l i bvi rtd .

Important

Since each of these steps significantly decreases the host's security protections against Q EMU guest domains, this configuration should only be used if there is no alternative to using .

Note

For more information on SELinux, refer to the Red Hat Enterprise Linux 6 Security-Enhanced Linux

User Guide.

B.13. Migrat ion fails wit h

Erro r: unabl e to reso l ve ad d ress

Symp t o m

Q EMU guest migration fails and this error message appears:

# virsh migrate qemu qemu+tcp://192.168.122.12/system

error: Unable to resolve address name_of_host service '49155':

Name or service not known

For example, if the destination hostname is "newyork", the error message will appear as:

# virsh migrate qemu qemu+tcp://192.168.122.12/system error: Unable to resolve address 'newyork' service '49155': Name or service not known

However, this error looks strange as we did not use "newyork" hostname anywhere.

In vest ig at io n

During migration, lib virt d running on the destination host creates a URI from an address and port where it expects to receive migration data and sends it back to lib virt d running on the source host.

In this case, the destination host (19 2. 16 8. 122. 12) has its name set to 'newyork'. For some reason, lib virt d running on that host is unable to resolve the name to an IP address that could be sent back and still be useful. For this reason, it returned the 'newyork' hostname hoping the source lib virt d would be more successful with resolving the name.

This can happen if DNS is not properly configured or /etc/ho sts has the hostname associated with local loopback address (127. 0 . 0 . 1).

14 4

⁠Appendix B. Common libvirt errors and t roubleshoot ing

Note that the address used for migration data cannot be automatically determined from the address used for connecting to destination lib virt d (for example, from

q emu+ tcp: //19 2. 16 8. 122. 12/system). This is because to communicate with the destination lib virt d , the source lib virt d may need to use network infrastructure different from that which virsh (possibly running on a separate machine) requires.

So lu t io n

The best solution is to configure DNS correctly so that all hosts involved in migration are able to resolve all host names.

If DNS cannot be configured to do this, a list of every host used for migration can be added manually to the /etc/ho sts file on each of the hosts. However, it is difficult to keep such lists consistent in a dynamic environment.

If the host names cannot be made resolvable by any means, vi rsh mi g rate supports specifying the migration host:

# virsh migrate qemu qemu+tcp://192.168.122.12/system tcp://192.168.122.12

Destination lib virt d will take the tcp: //19 2. 16 8. 122. 12 URI and append an automatically generated port number. If this is not desirable (because of firewall configuration, for example), the port number can be specified in this command:

# virsh migrate qemu qemu+tcp://192.168.122.12/system tcp://192.168.122.12:12345

Another option is to use tunnelled migration. Tunnelled migration does not create a separate connection for migration data, but instead tunnels the data through the connection used for communication with destination lib virt d (for example,

q emu+ tcp: //19 2. 16 8. 122. 12/system):

# virsh migrate qemu qemu+tcp://192.168.122.12/system --p2p -tunnelled

B.14 . Migrat ion fails wit h

Unabl e to al l o w access fo r d i sk path: No such fi l e o r d i recto ry

Symp t o m

A guest virtual machine (or domain) cannot be migrated because lib virt cannot access the disk image(s):

# virsh migrate qemu qemu+tcp://name_of_host/system error: Unable to allow access for disk path

/var/lib/libvirt/images/qemu.img: No such file or directory

For example, if the destination hostname is "newyork", the error message will appear as:

# virsh migrate qemu qemu+tcp://newyork/system error: Unable to allow access for disk path

/var/lib/libvirt/images/qemu.img: No such file or directory

14 5

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

In vest ig at io n

By default, migration only transfers the in-memory state of a running guest (such as memory or CPU state). Although disk images are not transferred during migration, they need to remain accessible at the same path by both hosts.

So lu t io n

Set up and mount shared storage at the same location on both hosts. The simplest way to do this is to use NFS:

Pro ced u re B.5. Set t in g u p sh ared st o rag e

1. Set up an NFS server on a host serving as shared storage. The NFS server can be one of the hosts involved in the migration, as long as all hosts involved are accessing the shared storage through NFS.

# mkdir -p /exports/images

# cat >>/etc/exports <<EOF

/exports/images 192.168.122.0/24(rw,no_root_squash)

EOF

2. Mount the exported directory at a common location on all hosts running lib virt . For example, if the IP address of the NFS server is 192.168.122.1, mount the directory with the following commands:

# cat >>/etc/fstab <<EOF

192.168.122.1:/exports/images /var/lib/libvirt/images nfs auto 0 0

EOF

# mount /var/lib/libvirt/images

Note

It is not possible to export a local directory from one host using NFS and mount it at the same path on another host — the directory used for storing disk images must be mounted from shared storage on both hosts. If this is not configured correctly, the guest virtual machine may lose access to its disk images during migration, because the source host's lib virt daemon may change the owner, permissions, and SELinux labels on the disk images after it successfully migrates the guest to its destination.

If lib virt detects that the disk images are mounted from a shared storage location, it will not make these changes.

B.15. No guest virt ual machines are present when libvirt d is st art ed

Symp t o m

The lib virt daemon is successfully started, but no guest virtual machines appear to be present.

14 6

⁠Appendix B. Common libvirt errors and t roubleshoot ing

# virsh list --all

Id Name State

----------------------------------------------------

In vest ig at io n

There are various possible causes of this problem. Performing these tests will help to determine the cause of this situation:

Verif y KVM kern el mo d u les

Verify that KVM kernel modules are inserted in the kernel:

# lsmod | grep kvm kvm_intel 121346 0 kvm 328927 1 kvm_intel

If you are using an AMD machine, verify the kvm_amd kernel modules are inserted in the kernel instead, using the similar command l smo d | g rep kvm_amd in the root shell.

If the modules are not present, insert them using the mo d pro be <modulename> command.

Note

Although it is uncommon, KVM virtualization support may be compiled into the kernel. In this case, modules are not needed.

Verif y virt u aliz at io n ext en sio n s

Verify that virtualization extensions are supported and enabled on the host:

# egrep "(vmx|svm)" /proc/cpuinfo flags : fpu vme de pse tsc ... svm ... skinit wdt npt lbrv svm_lock nrip_save flags : fpu vme de pse tsc ... svm ... skinit wdt npt lbrv svm_lock nrip_save

Enable virtualization extensions in your hardware's firmware configuration within the BIOS setup. Refer to your hardware documentation for further details on this.

Verif y clien t URI co n f ig u rat io n

Verify that the URI of the client is configured as desired:

# virsh uri vbox:///system

For example, this message shows the URI is connected to the Virt u alBo x hypervisor, not Q EMU, and reveals a configuration error for a URI that is otherwise set to connect to a Q EMU hypervisor. If the URI was correctly connecting to Q EMU, the same message would appear instead as:

14 7

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

# virsh uri qemu:///system

This situation occurs when there are other hypervisors present, which lib virt may speak to by default.

So lu t io n

After performing these tests, use the following command to view a list of guest virtual machines:

# virsh list --all

B.16. Unable t o connect t o server at 'host :16509': Connect ion refused

... error: failed t o connect t o t he hypervisor

Symp t o m

While lib virt d should listen on TCP ports for connections, the connections fail:

# virsh -c qemu+tcp://host/system error: unable to connect to server at 'host:16509': Connection refused error: failed to connect to the hypervisor

The lib virt daemon is not listening on TCP ports even after changing configuration in

/etc/l i bvi rt/l i bvi rtd . co nf:

# grep listen_ /etc/libvirt/libvirtd.conf

listen_tls = 1 listen_tcp = 1 listen_addr = "0.0.0.0"

However, the TCP ports for lib virt are still not open after changing configuration:

# netstat -lntp | grep libvirtd

In vest ig at io n

The lib virt daemon was started without the --l i sten option. Verify this by running this command:

# ps aux | grep libvirtd root 27314 0.0 0.0 1000920 18304 ? Sl Feb16 1:19 libvirtd --daemon

The output does not contain the --l i sten option.

So lu t io n

Start the daemon with the --l i sten option.

14 8

⁠Appendix B. Common libvirt errors and t roubleshoot ing

To do this, modify the /etc/sysco nfi g /l i bvi rtd file and uncomment the following line:

#LIBVIRTD_ARGS="--listen"

Then restart the lib virt d service with this command:

# /etc/init.d/libvirtd restart

B.17. Common XML errors

The lib virt tool uses XML documents to store structured data. A variety of common errors occur with

XML documents when they are passed to lib virt through the API. Several common XML errors — including misformatted XML, inappropriate values, and missing elements — are detailed below.

B.17.1. Edit ing domain definit ion

Although it is not recommended, it is sometimes necessary to edit a guest virtual machine's (or a domain's) XML file manually. To access the guest's XML for editing, use the following command:

# virsh edit name_of_guest.xml

This command opens the file in a text editor with the current definition of the guest virtual machine.

After finishing the edits and saving the changes, the XML is reloaded and parsed by lib virt . If the

XML is correct, the following message is displayed:

# virsh edit name_of_guest.xml

Domain name_of_guest.xml XML configuration edited.

Important

When using the ed i t command in virsh to edit an XML document, save all changes before exiting the editor.

After saving the XML file, use the xml l i nt command to validate that the XML is well-formed, or the

vi rt-xml -val i d ate command to check for usage problems:

# xmllint --noout config.xml

# virt-xml-validate config.xml

If no errors are returned, the XML description is well-formed and matches the lib virt schema. While the schema does not catch all constraints, fixing any reported errors will further troubleshooting.

XML d o cu men t s st o red b y lib virt

14 9

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

These documents contain definitions of states and configurations for the guests. These documents are automatically generated and should not be edited manually. Errors in these documents contain the file name of the broken document. The file name is valid only on the host machine defined by the URI, which may refer to the machine the command was run on.

Errors in files created by lib virt are rare. However, one possible source of these errors is a downgrade of lib virt — while newer versions of lib virt can always read XML generated by older versions, older versions of lib virt may be confused by XML elements added in a newer version.

B.17.2. XML synt ax errors

Syntax errors are caught by the XML parser. The error message contains information for identifying the problem.

This example error message from the XML parser consists of three lines — the first line denotes the error message, and the two following lines contain the context and location of the XML code containing the error. The third line contains an indicator showing approximately where the error lies on the line above it: error: (name_of_guest.xml):6: StartTag: invalid element name

<vcpu>2</vcpu><

-----------------^

In f o rmat io n co n t ain ed in t h is messag e:

( name_of_guest.xml)

This is the file name of the document that contains the error. File names in parentheses are symbolic names to describe XML documents parsed from memory, and do not directly correspond to files on disk. File names that are not contained in parentheses are local files that reside on the target of the connection.

This is the line number in the XML file that contains the error.

St art T ag : in valid elemen t n ame

This is the error message from the lib xml2 parser, which describes the specific

XML error.

B.1 7 .2 .1 . St ray < in t he do cum e nt

Symp t o m

The following error occurs: error: (name_of_guest.xml):6: StartTag: invalid element name

<vcpu>2</vcpu><

-----------------^

In vest ig at io n

This error message shows that the parser expects a new element name after the < symbol on line 6 of a guest's XML file.

150

⁠Appendix B. Common libvirt errors and t roubleshoot ing

Ensure line number display is enabled in your text editor. Open the XML file, and locate the text on line 6:

<name>name_of_guest</name>

<vcpu>2</vcpu><

This snippet of a guest's XML file contains an extra < in the document:

So lu t io n

Remove the extra < or finish the new element.

B.1 7 .2 .2 . Unt e rm inat e d at t ribut e

Symp t o m

The following error occurs: error: (name_of_guest.xml):2: Unescaped '<' not allowed in attributes values

<name>name_of_guest</name>

--^

In vest ig at io n

This snippet of a guest's XML file contains an unterminated element attribute value:

<name>name_of_guest</name>

In this case, ' kvm' is missing a second quotation mark. Strings of attribute values, such as quotation marks and apostrophes, must be opened and closed, similar to XML start and end tags.

So lu t io n

Correctly open and close all attribute value strings.

B.1 7 .2 .3. Ope ning and e nding t ag m ism at ch

Symp t o m

The following error occurs: error: (name_of_guest.xml):61: Opening and ending tag mismatch: clock line 16 and domain

</domain>

---------^

In vest ig at io n

The error message above contains three clues to identify the offending tag:

151

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

The message following the last colon, cl o ck l i ne 16 and d o mai n, reveals that

<cl o ck> contains a mismatched tag on line 16 of the document. The last hint is the pointer in the context part of the message, which identifies the second offending tag.

Unpaired tags must be closed with />. The following snippet does not follow this rule and has produced the error message shown above:

...

This error is caused by mismatched XML tags in the file. Every XML tag must have a matching start and end tag.

O t h er examp les o f mismat ch ed XML t ag s

The following examples produce similar error messages and show variations of mismatched XML tags.

This snippet contains an unended pair tag for <features>:

...

<acpi/>

<pae/>

...

</domain>

This snippet contains an end tag (</name>) without a corresponding start tag:

</name>

...

</domain>

So lu t io n

Ensure all XML tags start and end correctly.

B.1 7 .2 .4 . T ypo graphical e rro rs in t ags

Symp t o m

The following error message appears: error: (name_of_guest.xml):1: Specification mandate value for attribute ty

-----------^

In vest ig at io n

XML errors are easily caused by a simple typographical error. This error message highlights the XML error — in this case, an extra white space within the word type — with a pointer.

152

⁠Appendix B. Common libvirt errors and t roubleshoot ing

These XML examples will not parse correctly because of typographical errors such as a missing special character, or an additional character:

<dom#ain type='kvm'>

So lu t io n

To identify the problematic tag, read the error message for the context of the file, and locate the error with the pointer. Correct the XML and save the changes.

B.17.3. Logic and configurat ion errors

A well-formatted XML document can contain errors that are correct in syntax but lib virt cannot parse.

Many of these errors exist, with two of the most common cases outlined below.

B.1 7 .3.1 . Vanishing part s

Symp t o m

Parts of the change you have made do not show up and have no effect after editing or defining the domain. The d efi ne or ed i t command works, but when dumping the XML once again, the change disappears.

In vest ig at io n

This error likely results from a broken construct or syntax that libvirt does not parse. The

lib virt tool will generally only look for constructs it knows but ignore everything else, resulting in some of the XML changes vanishing after lib virt parses the input.

So lu t io n

Validate the XML input before passing it to the ed i t or d efi ne commands. The lib virt developers maintain a set of XML schemas bundled with lib virt which define the majority of the constructs allowed in XML documents used by lib virt .

Validate lib virt XML files using the following command:

# virt-xml-validate libvirt.xml

If this command passes, lib virt will likely understand all constructs from your XML, except if the schemas cannot detect options which are valid only for a given hypervisor. Any XML generated by lib virt as a result of a vi rsh d ump command, for example, should validate without error.

B.1 7 .3.2 . Inco rre ct drive de vice t ype

Symp t o m

The definition of the source image for the CD-ROM virtual drive is not present, despite being added:

153

Virt ualizat ion Host Configurat ion and G uest Inst allat ion G uide

# virsh dumpxml domain

...

</disk>

...

</domain>

So lu t io n

Correct the XML by adding the missing <so urce> parameter as follows:

</disk>

A type= ' bl o ck' disk device expects that the source is a physical device. To use the disk with an image file, use type= ' fi l e' instead.

154

Appendix C. Revision History

Revisio n 0.5- 4 2 Mo n May 02 2016

Multiple updates for the 6.8 GA release

Revisio n 0.5- 4 1 T u e Mar 01 2016

Multiple updates for the 6.8 beta release

Revisio n 0.5- 4 0

Cleaned up the Revision History

T h u O ct 08 2015

Revisio n 0.5- 39

Republishing for 6.7 Beta.

Wed Ap r 29 2015

⁠Appendix C. Revision Hist ory

Jiri Herrman n

Dayle Parker

155

जाहिरात

Red Hat Enterprise Linux 6

Virtualization Host Configuration and

Guest Installation Guide

Installing and configuring your virtual environment

Jiri Herrmann

Laura Bailey

Tahlia Richardson

Scott Radvan

Dayle Parker

Red Hat Enterprise Linux 6 Virtualization Host Configuration and Guest

Installation Guide

Installing and configuring your virtual environment

Jiri Herrmann

Red Hat Custo mer Co ntent Services [email protected] m

Tahlia Richardso n

Red Hat Custo mer Co ntent Services

Dayle Parker

Red Hat Custo mer Co ntent Services

Laura Bailey

Red Hat Custo mer Co ntent Services

Sco tt Radvan

Red Hat Custo mer Co ntent Services

Legal Notice

Co pyright © 20 16 Red Hat, Inc.

This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0

Unpo rted License . If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red

Hat trademarks must be remo ved.

Red Hat, as the licenso r o f this do cument, waives the right to enfo rce, and agrees no t to assert,

Sectio n 4 d o f CC-BY-SA to the fullest extent permitted by applicable law.

Red Hat, Red Hat Enterprise Linux, the Shado wman lo go , JBo ss, MetaMatrix, Fedo ra, the Infinity

Lo go , and RHCE are trademarks o f Red Hat, Inc., registered in the United States and o ther co untries.

Linux ® is the registered trademark o f Linus To rvalds in the United States and o ther co untries.

Java ® is a registered trademark o f Oracle and/o r its affiliates.

XFS ® is a trademark o f Silico n Graphics Internatio nal Co rp. o r its subsidiaries in the United

States and/o r o ther co untries.

MySQL ® is a registered trademark o f MySQL AB in the United States, the Euro pean Unio n and o ther co untries.

No de.js ® is an o fficial trademark o f Jo yent. Red Hat So ftware Co llectio ns is no t fo rmally related to o r endo rsed by the o fficial Jo yent No de.js o pen so urce o r co mmercial pro ject.

All o ther trademarks are the pro perty o f their respective o wners.

Abstract

This guide co vers KVM packages, co mpatibility and restrictio ns. Also included are ho st co nfiguratio n details and instructio ns fo r installing guest virtual machines o f different types, PCI device co nfiguratio n and SR-IOV.

Table of Contents

Chapter 1. Introduction

1.1. What is in t his guide?

1.2. Document at ion Suit e

Chapter 2. System requirements

Chapter 3. KVM guest virtual machine compatibility

3.1. Red Hat Ent erprise Linux 6 support limit s

3.2. Support ed CPU Models

Chapter 4. Virtualization restrictions

4 .1. KVM rest rict ions

4 .2. Applicat ion rest rict ions

4 .3. Ot her rest rict ions

Chapter 5. Installing the virtualization packages

5.1. Configuring a Virt ualizat ion Host inst allat ion

5.2. Inst alling virt ualizat ion packages on an exist ing Red Hat

Ent erprise Linux syst em

5.3. Regist ering t he hypervisor and virt ual machine

5.3. Regist ering t he hypervisor and virt ual machine

5.3.1. Inst alling virt -who on t he host physical machine

5.3.2. Regist ering a new guest virt ual machine

5.3.3. Removing a guest virt ual machine ent ry

5.3.4 . Inst alling virt -who manually

5.3.5. T roubleshoot ing virt -who

Chapter 6. Guest virtual machine installation overview

6.1. Guest virt ual machine prerequisit es and considerat ions

6.2. Creat ing guest s wit h virt -inst all

6.3. Creat ing guest s wit h virt -manager

6.4 . Inst alling guest virt ual machines wit h PXE

Chapter 7. Installing a Red Hat Enterprise Linux 6 guest virtual machine on a Red Hat Enterprise Linux 6 host

7.1. Creat ing a Red Hat Ent erprise Linux 6 guest wit h local inst allat ion media

7.2. Creat ing a Red Hat Ent erprise Linux 6 guest wit h a net work inst allat ion t ree

7.3. Creat ing a Red Hat Ent erprise Linux 6 guest wit h PXE

Chapter 8. Virtualizing Red Hat Enterprise Linux on Other

Platforms

8.1. On VMware ESX

8.2. On Hyper-V

Chapter 9. Installing a fully-virtualized Windows guest

9.1. Using virt -inst all t o creat e a guest

Chapter 10. KVM Para-virtualized (virtio) Drivers

10.1. Inst alling t he KVM Windows virt io drivers

**B.7. Virt ual net work default has not been st art ed**