Red Hat Enterprise Linux Virtualization Administration Guide
Pubblicità
Pubblicità
Red Hat Enterprise Linux 6
Virtualization Administration Guide
Managing your virtual environment
Laura Novich Scott Radvan
Red Hat Enterprise Linux 6 Virtualization Administration Guide
Managing your virtual environment
Laura No vich
Red Hat Custo mer Co ntent Services lno [email protected] m
Sco tt Radvan
Red Hat Custo mer Co ntent Services [email protected] m
Legal Notice
Co pyright © 20 13 Red Hat, Inc.
This do cument is licensed by Red Hat under the
Creative Co mmo ns Attributio n-ShareAlike 3.0
Unpo rted License
. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red
Hat trademarks must be remo ved.
Red Hat, as the licenso r o f this do cument, waives the right to enfo rce, and agrees no t to assert,
Sectio n 4 d o f CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shado wman lo go , JBo ss, MetaMatrix, Fedo ra, the Infinity
Lo go , and RHCE are trademarks o f Red Hat, Inc., registered in the United States and o ther co untries.
Linux ® is the registered trademark o f Linus To rvalds in the United States and o ther co untries.
Java ® is a registered trademark o f Oracle and/o r its affiliates.
XFS ® is a trademark o f Silico n Graphics Internatio nal Co rp. o r its subsidiaries in the United
States and/o r o ther co untries.
MySQL ® is a registered trademark o f MySQL AB in the United States, the Euro pean Unio n and o ther co untries.
No de.js ® is an o fficial trademark o f Jo yent. Red Hat So ftware Co llectio ns is no t fo rmally related to o r endo rsed by the o fficial Jo yent No de.js o pen so urce o r co mmercial pro ject.
The OpenStack ® Wo rd Mark and OpenStack Lo go are either registered trademarks/service marks o r trademarks/service marks o f the OpenStack Fo undatio n, in the United States and o ther co untries and are used with the OpenStack Fo undatio n's permissio n. We are no t affiliated with, endo rsed o r spo nso red by the OpenStack Fo undatio n, o r the OpenStack co mmunity.
All o ther trademarks are the pro perty o f their respective o wners.
Abstract
The Virtualizatio n Administratio n Guide co vers administratio n o f ho st physical machines, netwo rking, sto rage, device and guest virtual machine management, and tro ublesho o ting.
T able of Cont ent s
1.1. Typ o g rap hic Co nventio ns
1.2. Pull-q uo te Co nventio ns
2. G etting Help and G iving Feed b ack
Table of Contents
1.1. Virtualizatio n Do cumentatio n Suite
3.1. Sto rag e security issues
3.2. SELinux and virtualizatio n
3.4. Virtualizatio n firewall info rmatio n
4.1. Security and Virtualizatio n
5.1. Live mig ratio n req uirements
5.2. Live mig ratio n and Red Hat Enterp rise Linux versio n co mp atib ility
5.3. Shared sto rag e examp le: NFS fo r a simp le mig ratio n
5.4. Live KVM mig ratio n with virsh
5.4.1. Ad d itio nal tip s fo r mig ratio n with virsh
5.4.2. Ad d itio nal o p tio ns fo r the virsh mig rate co mmand
5.5. Mig rating with virt-manag er
6 .1. Remo te manag ement with SSH
6 .2. Remo te manag ement o ver TLS and SSL
7.2. O verco mmitting virtualized CPUs
9 .1. Co ntro l G ro up s (cg ro up s)
9 .3. Running Red Hat Enterp rise Linux as a g uest virtual machine o n a Hyp er-V hyp erviso r
9 .4. G uest virtual machine memo ry allo catio n
9 .5. Auto matically starting g uest virtual machines
9 .6 . Disab le SMART d isk mo nito ring fo r g uest virtual machines
9 .7. Co nfig uring a VNC Server
9 .8 . G enerating a new uniq ue MAC ad d ress
9 .8 .1. Ano ther metho d to g enerate a new MAC fo r yo ur g uest virtual machine
9 .9 . Imp ro ving g uest virtual machine resp o nse time
58
58
58
59
6 0
6 0
6 0
6 1
6 1
6 2
1
14
14
14
15
12
12
13
16
24
25
26
28
29
30
32
33
35
42
44
47
51
52
19
19
21
21
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
9 .9 . Imp ro ving g uest virtual machine resp o nse time
9 .10 . Virtual machine timer manag ement with lib virt
9 .10 .1. timer child element fo r clo ck
9 .10 .4. freq uency, mo d e, and p resent
9 .10 .5. Examp les using clo ck synchro nizatio n
9 .11. Using PMU to mo nito r g uest virtual machine p erfo rmance
9 .12. G uest virtual machine p o wer manag ement
10 .1.1. Assig ning a PCI d evice with virsh
10 .1.2. Assig ning a PCI d evice with virt-manag er
10 .1.3. PCI d evice assig nment with virt-install
10 .1.4. Detaching an assig ned PCI d evice
10 .1.5. Creating PCI b rid g es
10 .1.7. Co nfig uring PCI assig nment (p assthro ug h) with SR-IO V d evices
10 .1.8 . Setting PCI d evice assig nment fro m a p o o l o f SR-IO V virtual functio ns
10 .2.1. Assig ning USB d evices to g uest virtual machines
10 .2.2. Setting a limit o n USB d evice red irectio n
10 .3. Co nfig uring d evice co ntro llers
10 .4. Setting ad d resses fo r d evices
10 .5. Manag ing sto rag e co ntro llers in a g uest virtual machine
10 .6 . Rand o m numb er g enerato r (RNG ) d evice
11.2.1. Q EMU g uest virtual machine ag ent p ro to co l
9 4
9 8
9 8
9 8
11.2.3. g uest-sync-d elimited
11.2.4. Creating a g uest virtual machine d isk b ackup
9 9
10 0
11.3. Setting a limit o n d evice red irectio n
10 2
8 4
8 5
8 9
9 0
9 2
78
79
8 1
8 4
8 4
6 7
6 9
72
74
77
78
6 5
6 5
6 6
6 6
6 2
6 2
6 4
6 4
6 4
2
13.1. Disk-b ased sto rag e p o o ls
13.1.1. Creating a d isk b ased sto rag e p o o l using virsh
13.1.2. Deleting a sto rag e p o o l using virsh
13.2. Partitio n-b ased sto rag e p o o ls
13.2.1. Creating a p artitio n-b ased sto rag e p o o l using virt-manag er
13.2.2. Deleting a sto rag e p o o l using virt-manag er
13.2.3. Creating a p artitio n-b ased sto rag e p o o l using virsh
13.2.4. Deleting a sto rag e p o o l using virsh
13.3. Directo ry-b ased sto rag e p o o ls
13.3.1. Creating a d irecto ry-b ased sto rag e p o o l with virt-manag er
13.3.2. Deleting a sto rag e p o o l using virt-manag er
10 5
10 6
10 8
10 9
111
112
112
115
116
118
119
119
122
13.3.2. Deleting a sto rag e p o o l using virt-manag er
13.3.3. Creating a d irecto ry-b ased sto rag e p o o l with virsh
13.3.4. Deleting a sto rag e p o o l using virsh
13.4. LVM-b ased sto rag e p o o ls
13.4.1. Creating an LVM-b ased sto rag e p o o l with virt-manag er
13.4.2. Deleting a sto rag e p o o l using virt-manag er
13.4.3. Creating an LVM-b ased sto rag e p o o l with virsh
13.4.4. Deleting a sto rag e p o o l using virsh
13.5. iSCSI-b ased sto rag e p o o ls
13.5.1. Co nfig uring a so ftware iSCSI targ et
13.5.2. Ad d ing an iSCSI targ et to virt-manag er
13.5.3. Deleting a sto rag e p o o l using virt-manag er
13.5.4. Creating an iSCSI-b ased sto rag e p o o l with virsh
13.5.5. Deleting a sto rag e p o o l using virsh
13.6 . NFS-b ased sto rag e p o o ls
13.6 .1. Creating a NFS-b ased sto rag e p o o l with virt-manag er
13.6 .2. Deleting a sto rag e p o o l using virt-manag er
13.7. G lusterFS sto rag e p o o ls
13.7.1. Creating a G lusterFS sto rag e p o o l using virsh
13.7.2. Deleting a G lusterFS sto rag e p o o l using virsh
14.3. Ad d ing sto rag e d evices to g uests
14.3.1. Ad d ing file b ased sto rag e to a g uest
14.3.2. Ad d ing hard d rives and o ther b lo ck d evices to a g uest
14.4. Deleting and remo ving vo lumes
15.1.6 . Disp laying b asic info rmatio n
15.2. Attaching and up d ating a d evice with virsh
15.3. Attaching interface d evices
15.4. Chang ing the med ia o f a CDRO M
15.5.1. Co nfig uring a d o main to b e started auto matically at b o o t
15.5.2. Co nnecting the serial co nso le fo r the g uest virtual machine
15.5.3. Defining a d o main with an XML file
15.5.4. Ed iting and d isp laying a d escrip tio n and title o f a d o main
15.5.5. Disp laying d evice b lo ck statistics
15.5.6 . Retrieving netwo rk statistics
15.5.7. Mo d ifying the link state o f a d o main's virtual interface
15.5.8 . Listing the link state o f a d o main's virtual interface
15.5.9 . Setting netwo rk interface b and wid th p arameters
15.5.10 . Retrieving memo ry statistics fo r a running d o main
15.5.11. Disp laying erro rs o n b lo ck d evices
15.5.12. Disp laying the b lo ck d evice size
15.5.13. Disp laying the b lo ck d evices asso ciated with a d o main
T able of Cont ent s
144
144
144
147
148
134
134
138
141
142
148
150
122
123
125
125
126
131
132
134
16 2
16 2
16 3
16 3
16 3
16 0
16 0
16 1
16 2
16 2
158
158
159
159
159
159
16 0
16 4
16 4
16 4
16 4
16 5
16 5
16 5
16 5
3
151
151
152
152
155
156
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
4
15.5.13. Disp laying the b lo ck d evices asso ciated with a d o main
15.5.14. Disp laying virtual interfaces asso ciated with a d o main
15.5.15. Using b lo ckco mmit to sho rten a b acking chain
15.5.16 . Using b lo ckp ull to sho rten a b acking chain
15.5.17. Using b lo ckresize to chang e the size o f a d o main p ath
15.5.18 . Disk imag e manag ement with live b lo ck co p y
15.5.19 . Disp laying a URI fo r co nnectio n to a g rap hical d isp lay
15.5.20 . Do main Retrieval Co mmand s
15.5.21. Co nverting Q EMU arg uments to d o main XML
15.5.22. Creating a d ump file o f a d o main's co re
15.5.23. Creating a virtual machine XML d ump (co nfig uratio n file)
15.5.24. Creating a g uest virtual machine fro m a co nfig uratio n file
15.6 . Ed iting a g uest virtual machine's co nfig uratio n file
15.6 .1. Ad d ing multifunctio n PCI d evices to KVM g uest virtual machines
15.6 .2. Sto p p ing a running d o main in o rd er to restart it later
15.6 .3. Disp laying CPU statistics fo r a sp ecified d o main
15.6 .4. Saving a screensho t
15.6 .5. Send ing a keystro ke co mb inatio n to a sp ecified d o main
15.6 .6 . Send ing p ro cess sig nal names to virtual p ro cesses
15.6 .7. Disp laying the IP ad d ress and p o rt numb er fo r the VNC d isp lay
15.7. NUMA no d e manag ement
15.7.1. Disp laying no d e info rmatio n
15.7.2. Setting NUMA p arameters
15.7.3. Disp laying the amo unt o f free memo ry in a NUMA cell
15.7.4. Disp laying a CPU list
15.7.5. Disp laying CPU statistics
15.7.6 . Susp end ing the ho st p hysical machine
15.7.7. Setting and d isp laying the no d e memo ry p arameters
15.7.8 . Creating d evices o n ho st no d es
15.7.9 . Detaching a no d e d evice
15.7.11. List d evices o n a no d e
15.7.12. Trig g ering a reset fo r a no d e
15.8 . Starting , susp end ing , resuming , saving and resto ring a g uest virtual machine
15.8 .1. Starting a d efined d o main
15.8 .2. Susp end ing a g uest virtual machine
15.8 .3. Susp end ing a running d o main
15.8 .4. Waking up a d o main fro m p msusp end state
15.8 .5. Und efining a d o main
15.8 .6 . Resuming a g uest virtual machine
15.8 .7. Save a g uest virtual machine
15.8 .8 . Up d ating the d o main XML file that will b e used fo r resto ring the g uest
15.8 .9 . Extracting the d o main XML file
15.8 .10 . Ed it Do main XML co nfig uratio n files
15.8 .11. Resto re a g uest virtual machine
15.9 . Shutting d o wn, reb o o ting and fo rce-shutd o wn o f a g uest virtual machine
15.9 .1. Shut d o wn a g uest virtual machine
15.9 .3. Manip ulating the lib virt-g uests co nfig uratio n setting s
18 6
15.9 .4. Reb o o ting a g uest virtual machine
15.9 .5. Fo rcing a g uest virtual machine to sto p
15.9 .6 . Resetting a virtual machine
15.10 . Retrieving g uest virtual machine info rmatio n
15.10 .1. G etting the d o main ID o f a g uest virtual machine
18 1
18 1
18 1
18 2
18 2
18 0
18 0
18 0
18 0
18 0
178
179
179
179
18 0
177
177
178
178
178
18 2
18 3
18 3
18 3
18 3
18 4
18 4
175
175
175
176
177
172
173
174
174
175
16 8
16 9
170
170
171
16 5
16 5
16 6
16 6
16 8
18 4
18 9
18 9
18 9
18 9
18 9
15.10 .1. G etting the d o main ID o f a g uest virtual machine
15.10 .2. G etting the d o main name o f a g uest virtual machine
15.10 .3. G etting the UUID o f a g uest virtual machine
15.10 .4. Disp laying g uest virtual machine info rmatio n
15.11. Sto rag e p o o l co mmand s
15.11.1. Searching fo r a sto rag e p o o l XML
15.11.2. Creating , d efining , and starting sto rag e p o o ls
15.11.2.1. Build ing a sto rag e p o o l
15.11.2.2. Creating and d efining a sto rag e p o o l fro m an XML file
15.11.2.3. Creating and starting a sto rag e p o o l fro m raw p arameters
15.11.2.4. Auto -starting a sto rag e p o o l
15.11.3. Sto p p ing and d eleting sto rag e p o o ls
15.11.4. Creating an XML d ump file fo r a p o o l
15.11.5. Ed iting the sto rag e p o o l's co nfig uratio n file
15.11.6 . Co nverting sto rag e p o o ls
15.12. Sto rag e Vo lume Co mmand s
15.12.1. Creating sto rag e vo lumes
15.12.1.1. Creating a sto rag e vo lume fro m an XML file
15.12.1.2. Clo ning a sto rag e vo lume
15.12.2. Deleting sto rag e vo lumes
15.12.3. Dump ing sto rag e vo lume info rmatio n to an XML file
15.12.4. Listing vo lume info rmatio n
15.12.5. Retrieving sto rag e vo lume info rmatio n
15.12.6 . Up lo ad ing and d o wnlo ad ing sto rag e vo lumes
15.12.6 .1. Up lo ad ing co ntents to a sto rag e vo lume
15.12.6 .2. Do wnlo ad ing the co ntents fro m a sto rag e vo lume
15.12.7. Re-sizing sto rag e vo lumes
15.13. Disp laying p er-g uest virtual machine info rmatio n
15.13.1. Disp laying the g uest virtual machines
15.13.2. Disp laying virtual CPU info rmatio n
15.13.3. Co nfig uring virtual CPU affinity
15.13.4. Disp laying info rmatio n ab o ut the virtual CPU co unts o f a g iven d o mian
15.13.5. Co nfig uring virtual CPU affinity
15.13.6 . Co nfig uring virtual CPU co unt
15.13.7. Co nfig uring memo ry allo catio n
15.13.8 . Chang ing the memo ry allo catio n fo r the d o main
15.13.9 . Disp laying g uest virtual machine b lo ck d evice info rmatio n
15.13.10 . Disp laying g uest virtual machine netwo rk d evice info rmatio n
15.14. Manag ing virtual netwo rks
15.15. Mig rating g uest virtual machines with virsh
15.15.1. Interface Co mmand s
15.15.1.1. Defining and starting a ho st p hysical machine interface via an XML file
15.15.1.2. Ed iting the XML co nfig uratio n file fo r the ho st interface
15.15.1.3. Listing active ho st interfaces
15.15.1.4. Co nverting a MAC ad d ress into an interface name
15.15.1.5. Sto p p ing a sp ecific ho st p hysical machine interface
15.15.1.6 . Disp laying the ho st co nfig uratio n file
15.15.1.7. Creating b rid g e d evices
15.15.1.8 . Tearing d o wn a b rid g e d evice
15.15.1.9 . Manip ulating interface snap sho ts
15.15.2. Manag ing snap sho ts
15.15.2.1. Creating Snap sho ts
15.15.2.2. Creating a snap sho t fo r the current d o main
15.15.2.3. Taking a snap sho t o f the current d o main
T able of Cont ent s
19 8
19 9
19 9
20 1
20 2
19 6
19 6
19 6
19 7
19 8
19 5
19 5
19 5
19 5
19 5
19 3
19 3
19 4
19 4
19 4
19 2
19 2
19 2
19 2
19 3
19 1
19 1
19 1
19 2
19 2
18 9
18 9
19 0
19 0
19 0
19 0
20 4
20 4
20 5
20 5
20 5
20 5
20 6
20 7
20 4
20 4
20 4
20 4
20 4
20 2
20 2
20 2
20 3
20 3
5
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
15.15.2.3. Taking a snap sho t o f the current d o main
15.15.2.4. snap sho t-ed it-d o main
15.15.2.5. snap sho t-info -d o main
15.15.2.6 . snap sho t-list-d o main
15.15.2.7. snap sho t-d ump xml d o main snap sho t
15.15.2.8 . snap sho t-p arent d o main
15.15.2.9 . snap sho t-revert d o main
15.15.2.10 . snap sho t-d elete d o main
15.16 . G uest virtual machine CPU mo d el co nfig uratio n
15.16 .2. Learning ab o ut the ho st p hysical machine CPU mo d el
15.16 .3. Determining a co mp atib le CPU mo d el to suit a p o o l o f ho st p hysical machines
15.17. Co nfig uring the g uest virtual machine CPU mo d el
15.18 . Manag ing reso urces fo r g uest virtual machines
15.19 . Setting sched ule p arameters
15.21. Disp lay o r set b lo ck I/O p arameters
15.22. Co nfig uring memo ry Tuning
15.23. Virtual Netwo rking Co mmand s
15.23.1. Auto starting a virtual netwo rk
15.23.2. Creating a virtual netwo rk fro m an XML file
15.23.3. Defining a virtual netwo rk fro m an XML file
15.23.4. Sto p p ing a virtual netwo rk
15.23.5. Creating a d ump file
15.23.6 . Ed ing a virtual netwo rk's XML co nfig uratio n file
15.23.7. G etting info rmatio n ab o ut a virtual netwo rk
15.23.8 . Listing info rmatio n ab o ut a virtual netwo rk
15.23.9 . Co nverting a netwo rk UUID to netwo rk name
15.23.10 . Starting a (p revio usly d efined ) inactive netwo rk
15.23.11. Und efining the co nfig uratio n fo r an inactive netwo rk
15.23.12. Co nverting a netwo rk name to netwo rk UUID
15.23.13. Up d ating an existing netwo rk d efinitio n file
16 .1. Starting virt-manag er
16 .2. The Virtual Machine Manag er main wind o w
16 .3. The virtual hard ware d etails wind o w
16 .3.1. Attaching USB d evices to a g uest virtual machine
16 .4. Virtual Machine g rap hical co nso le
16 .5. Ad d ing a remo te co nnectio n
16 .6 . Disp laying g uest d etails
16 .7. Perfo rmance mo nito ring
16 .8 . Disp laying CPU usag e fo r g uests
16 .9 . Disp laying CPU usag e fo r ho sts
16 .10 . Disp laying Disk I/O
16 .11. Disp laying Netwo rk I/O
6
17.4.1. Viewing file systems with g uestfish
17.4.1.1. Manual listing and viewing
243
243
244
244
245
245
227
228
235
236
237
220
221
221
223
225
238
239
217
217
217
217
217
216
216
216
216
216
218
218
218
218
218
213
213
214
215
216
20 9
210
210
210
210
20 7
20 7
20 7
20 7
20 8
20 8
20 9
17.4.1.1. Manual listing and viewing
17.4.1.2. Via g uestfish insp ectio n
17.4.1.3. Accessing a g uest virtual machine b y name
17.4.2. Mo d ifying files with g uestfish
17.4.3. O ther actio ns with g uestfish
17.4.4. Shell scrip ting with g uestfish
17.4.5. Aug eas and lib g uestfs scrip ting
17.6 . virt-rescue: The rescue shell
17.7. virt-d f: Mo nito ring d isk usag e
17.8 . virt-resize: resizing g uest virtual machines o ffline
17.8 .2. Exp and ing a d isk imag e
17.9 . virt-insp ecto r: insp ecting g uest virtual machines
17.9 .3. Running virt-insp ecto r
17.10 . virt-win-reg : Read ing and ed iting the Wind o ws Reg istry
17.11. Using the API fro m Pro g ramming Lang uag es
17.11.1. Interactio n with the API via a C p ro g ram
17.13. Where to find further d o cumentatio n
19 .1. Virtual netwo rk switches
19 .2. Netwo rk Ad d ress Translatio n
19 .3. Netwo rking p ro to co ls
19 .4. The d efault co nfig uratio n
19 .5. Examp les o f co mmo n scenario s
19 .6 . Manag ing a virtual netwo rk
19 .7. Creating a virtual netwo rk
19 .8 . Attaching a virtual netwo rk to a g uest
19 .9 . Directly attaching to p hysical interface
19 .10 . Ap p lying netwo rk filtering
19 .10 .3. Filtering chain p rio rities
19 .10 .4. Usag e o f variab les in filters
T able of Cont ent s
251
251
252
252
252
249
249
249
250
251
245
246
247
247
247
248
248
256
256
256
257
258
254
254
254
254
256
26 2
26 2
26 3
26 4
274
275
28 1
28 5
28 7
271
271
272
273
273
26 6
26 7
26 8
26 8
26 9
270
28 7
28 8
29 0
29 0
7
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
19 .10 .4. Usag e o f variab les in filters
19 .10 .5. Auto matic IP ad d ress d etectio n and DHCP sno o p ing
19 .10 .5.2. DHCP sno o p ing
19 .10 .6 . Reserved Variab les
19 .10 .7. Element and attrib ute o verview
19 .10 .8 . References to o ther filters
19 .10 .10 . Sup p o rted p ro to co ls
19 .10 .10 .1. MAC (Ethernet)
19 .10 .10 .2. VLAN (8 0 2.1Q )
19 .10 .10 .3. STP (Sp anning Tree Pro to co l)
19 .10 .10 .9 . IG MP, ESP, AH, UDPLITE, 'ALL'
19 .10 .10 .10 . TCP/UDP/SCTP o ver IPV6
19 .10 .10 .12. IG MP, ESP, AH, UDPLITE, 'ALL' o ver IPv6
19 .10 .11. Ad vanced Filter Co nfig uratio n To p ics
19 .10 .11.1. Co nnectio n tracking
19 .10 .11.2. Limiting Numb er o f Co nnectio ns
19 .10 .11.3. Co mmand line to o ls
19 .10 .11.4. Pre-existing netwo rk filters
19 .10 .11.5. Writing yo ur o wn filters
19 .10 .11.6 . Samp le custo m filter
19 .11.1. Creating Multicast tunnels
19 .11.2. Creating TCP tunnels
19 .13. Ap p lying Q o S to yo ur virtual netwo rk
8
30 8
310
313
314
314
314
315
316
30 5
30 5
30 6
30 7
30 7
30 1
30 2
30 3
30 4
30 4
29 7
29 8
29 9
30 0
30 0
29 4
29 5
29 6
29 7
29 7
29 0
29 2
29 2
29 3
29 4
29 4
318
319
319
319
318
318
318
318
318
320
320
317
317
317
317
317
318
318
VG A card emulatio n
20 .7. Linux/Multib o o t b o o t
Disab le kernel mo d e PIT reinjectio n
Serial p o rt, mo nito r, Q MP
20 .9 . Help and info rmatio n o p tio ns
20 .10 . Miscellaneo us o p tio ns
21.1. G eneral info rmatio n and metad ata
21.2. O p erating system b o o ting
21.2.1. BIO S b o o tlo ad er
21.2.2. Ho st p hysical machine b o o tlo ad er
21.2.3. Direct kernel b o o t
21.3. SMBIO S system info rmatio n
21.10 . Reso urce p artitio ning
21.11. CPU mo d el and to p o lo g y
21.11.1. G uest virtual machine NUMA to p o lo g y
T able of Cont ent s
331
331
331
331
331
331
331
331
332
330
330
331
331
331
330
330
330
330
330
329
329
329
329
330
328
329
329
329
329
320
320
320
321
321
322
322
342
343
344
345
349
337
338
339
341
341
333
334
334
336
336
337
9
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
21.11.1. G uest virtual machine NUMA to p o lo g y
21.12. Events co nfig uratio n
21.16 .1. Hard d rives, flo p p y d isks, CDRO Ms
21.16 .1.7. Ad d itio nal Device Elements
21.16 .6 . Ho st p hysical machine d evice assig nment
21.16 .6 .1. USB / PCI d evices
21.16 .6 .2. Blo ck / character d evices
21.16 .7. Red irected d evices
21.16 .8 . Smartcard d evices
21.16 .9 . Netwo rk interfaces
21.16 .9 .1. Virtual netwo rks
21.16 .9 .3. Setting a p o rt masq uerad ing rang e
21.16 .9 .4. Usersp ace SLIRP stack
21.16 .9 .5. G eneric Ethernet co nnectio n
21.16 .9 .6 . Direct attachment to p hysical interfaces
21.16 .9 .7. PCI p assthro ug h
21.16 .9 .8 . Multicast tunnel
21.16 .9 .10 . Setting NIC d river-sp ecific o p tio ns
21.16 .9 .11. O verrid ing the targ et element
21.16 .9 .12. Sp ecifying b o o t o rd er
21.16 .9 .13. Interface RO M BIO S co nfig uratio n
21.16 .9 .14. Q uality o f service
21.16 .9 .15. Setting VLAN tag (o n sup p o rted netwo rk typ es o nly)
21.16 .9 .16 . Mo d ifying virtual link state
21.16 .12. G rap hical frameb uffers
21.16 .14. Co nso les, serial, p arallel, and channel d evices
21.16 .15. G uest virtual machine interfaces
21.16 .17. Ho st p hysical machine interface
21.19 . Memo ry b allo o n d evice
21.22. Examp le d o main XML co nfig uratio n
10
38 3
38 3
38 4
38 5
38 5
379
38 0
38 0
38 1
38 2
374
375
375
375
376
36 8
36 9
370
372
372
36 2
36 3
36 5
36 5
36 5
358
358
358
359
36 1
355
355
357
357
358
349
349
351
352
353
39 2
39 4
39 5
40 0
40 1
38 6
38 6
38 7
39 0
39 1
40 2
40 2
40 3
40 4
21.22. Examp le d o main XML co nfig uratio n
22.1. Deb ug g ing and tro ub lesho o ting to o ls
22.2. Creating virsh d ump files
22.4. G uest virtual machine fails to shutd o wn
22.5. Tro ub lesho o ting with serial co nso les
22.6 . Virtualizatio n lo g files
22.8 . Live Mig ratio n Erro rs
22.9 . Enab ling Intel VT-x and AMD-V virtualizatio n hard ware extensio ns in BIO S
22.10 . KVM netwo rking p erfo rmance
22.11. Wo rkaro und fo r creating external snap sho ts with lib virt
22.12. Missing characters o n g uest co nso le with Jap anese keyb o ard
22.13. Kno wn Wind o ws XP g uest issues
22.14. Verifying virtualizatio n extensio ns
T able of Cont ent s
40 4
413
413
414
414
415
40 7
40 8
40 9
412
413
416
417
417
418
B.2. Installed d o cumentatio n
421
421
11
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Preface
1. Document Convent ions
This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.
1.1. T ypographic Convent ions
Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows.
Mo no -spaced Bo l d
Used to highlight system input, including shell commands, file names and paths. Also used to highlight keys and key combinations. For example:
To see the contents of the file my_next_bestsel l i ng _no vel in your current working directory, enter the cat my_next_bestsel l i ng _no vel command at the shell prompt and press Enter to execute the command.
The above includes a file name, a shell command and a key, all presented in mono-spaced bold and all distinguishable thanks to context.
Key combinations can be distinguished from an individual key by the plus sign that connects each part of a key combination. For example:
Press Enter to execute the command.
Press C trl +Al t+F2 to switch to a virtual terminal.
The first example highlights a particular key to press. The second example highlights a key combination: a set of three keys pressed simultaneously.
If source code is discussed, class names, methods, functions, variable names and returned values mentioned within a paragraph will be presented as above, in mo no -spaced bo l d . For example:
File-related classes include fi l esystem for file systems, fi l e for files, and d i r for directories. Each class has its own associated set of permissions.
Pro p o rt io n al Bo ld
This denotes words or phrases encountered on a system, including application names; dialog-box text; labeled buttons; check-box and radio-button labels; menu titles and submenu titles. For example:
Choose Syst em → Pref eren ces → Mo u se from the main menu bar to launch
Mo u se Pref eren ces. In the Butto ns tab, select the Left-hand ed mo use check box and click C l o se to switch the primary mouse button from the left to the right
(making the mouse suitable for use in the left hand).
To insert a special character into a g ed it file, choose Ap p licat io n s →
Accesso ries → Ch aract er Map from the main menu bar. Next, choose Search →
Fin d … from the Ch aract er Map menu bar, type the name of the character in the
Search field and click Next. The character you sought will be highlighted in the
12
Preface
C haracter T abl e. Double-click this highlighted character to place it in the T ext
to co py field and then click the C o py button. Now switch back to your document and choose Ed it → Past e from the g ed it menu bar.
The above text includes application names; system-wide menu names and items; application-specific menu names; and buttons and text found within a GUI interface, all presented in proportional bold and all distinguishable by context.
Mono-spaced Bold Italic or Proportional Bold Italic
Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example:
To connect to a remote machine using ssh, type ssh username@ domain.name at a shell prompt. If the remote machine is exampl e. co m and your username on that machine is john, type ssh jo hn@ exampl e. co m.
The mo unt -o remo unt file-system command remounts the named file system.
For example, to remount the /ho me file system, the command is mo unt -o remo unt
/ho me.
To see the version of a currently installed package, use the rpm -q package command. It will return a result as follows: package-version-release.
Note the words in bold italics above: username, domain.name, file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system.
Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example:
Publican is a DocBook publishing system.
1.2. Pull-quot e Convent ions
Terminal output and source code listings are set off visually from the surrounding text.
Output sent to a terminal is set in mo no -spaced ro man and presented thus: books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs
Source-code listings are also set in mo no -spaced ro man but add syntax highlighting as follows: static int kvm_vm_ioctl_deassign_device( struct kvm *kvm,
struct kvm_assigned_pci_dev *assigned_dev)
{
int r = 0;
struct kvm_assigned_dev_kernel *match;
mutex_lock(& kvm->lock);
match = kvm_find_assigned_dev(& kvm->arch.assigned_dev_head,
assigned_dev->assigned_dev_id);
if (!match) {
printk(KERN_INFO
"%s: device hasn't been assigned
13
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
before, "
"so cannot be deassigned \n " , __func__);
r = -EINVAL;
goto out;
}
kvm_deassign_device(kvm, match);
kvm_free_assigned_device(kvm, match);
out:
mutex_unlock(& kvm->lock);
return r;
}
1.3. Not es and Warnings
Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.
Note
Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier.
Important
Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply. Ignoring a box labeled “ Important” will not cause data loss but may cause irritation and frustration.
Warning
Warnings should not be ignored. Ignoring warnings will most likely cause data loss.
2. Get t ing Help and Giving Feedback
2.1. Do You Need Help?
If you experience difficulty with a procedure described in this documentation, visit the Red Hat
Customer Portal at http://access.redhat.com
. Through the customer portal, you can: search or browse through a knowledgebase of technical support articles about Red Hat products.
submit a support case to Red Hat Global Support Services (GSS).
access other product documentation.
14
Preface
Red Hat also hosts a large number of electronic mailing lists for discussion of Red Hat software and technology. You can find a list of publicly available mailing lists at https://www.redhat.com/mailman/listinfo . Click on the name of any mailing list to subscribe to that list or to access the list archives.
2.2. We Need Feedback!
If you find a typographical error in this manual, or if you have thought of a way to make this manual better, we would love to hear from you! Please submit a report in Bugzilla: http://bugzilla.redhat.com/ against the product Do cu men t at io n .
When submitting a bug report, be sure to mention the manual's identifier: doc-
Virtualization_Administration_Guide
If you have a suggestion for improving the documentation, try to be as specific as possible when describing it. If you have found an error, please include the section number and some of the surrounding text so we can find it easily.
15
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 1. Introduction
1.1. Virt ualizat ion Document at ion Suit e
Red Hat offers a wealth of documentation solutions across its various virtualization products.
Coverage of Red Hat Enterprise Linux and its inbuilt virtualization products includes:
Red Hat Enterprise Linux — Virtualization Getting Started Guide: This guide provides an introduction to virtualization concepts, advantages, and tools, and an overview of Red Hat virtualization documentation and products.
Red Hat Enterprise Linux — Virtualization Host Configuration and Guest Installation Guide: This guide covers the installation of virtualization software and configuration of guest virtual machines on a host physical machine.
Red Hat Enterprise Linux — Virtualization Administration Guide: This guide covers administration of host physical machines, networking, storage, and device and guest virtual machine management using either virt-manager or virsh as primary configuration tools. This guide also includes a libvirt and QEMU reference, as well as troubleshooting information.
Red Hat Enterprise Linux — Virtualization Security Guide: This guide provides an overview of virtualization security technologies provided by Red Hat. Also included are recommendations for securing host physical machines, guest virtual machines, and shared infrastructure and resources in virtualized environments.
Red Hat Enterprise Linux — Virtualization Tuning and Optimization Guide: This guide provides tips, tricks and suggestions for making full use of virtualization performance features and options for your systems and guest virtual machines.
Red Hat Enterprise Linux — Hypervisor Deployment Guide describes how to deploy and install the
Red Hat Enterprise Virtualization Hypervisor. Read this guide if you need advanced information about installing and deploying Hypervisors. The basic installation of Hypervisor host physical machines is also described in the Red Hat Enterprise Virtualization Installation Guide.
Red Hat Enterprise Linux — V2V Guide describes importing virtual machines from KVM, Xen and
VMware ESX/ESX(i) hypervisors to Red Hat Enterprise Virtualization and KVM managed by libvirt.
The Red Hat Enterprise Virtualization documentation suite provides information on installation, development of applications, configuration and usage of the Red Hat Enterprise Virtualization platform and its related products.
Red Hat Enterprise Virtualization — Administration Guide describes how to set up, configure and manage Red Hat Enterprise Virtualization. It assumes that you have successfully installed the Red
Hat Enterprise Virtualization Manager and host physical machines.
Red Hat Enterprise Virtualization — Command Line Shell Guide contains information for installing and using the Red Hat Enterprise Virtualization Manager command line shell.
Red Hat Enterprise Virtualization — Developer Guide explains how to use the REST API. It covers the fundamentals of the REST architectural concepts in the context of a virtualization environment and provides examples of the API in operation. It also documents the installation and use of the
Python Software Development Kit.
Red Hat Enterprise Virtualization — Evaluation Guide enables prospective customers to evaluate the features of Red Hat Enterprise Virtualization. Use this guide if you have an evaluation license.
Red Hat Enterprise Virtualization — Installation Guide describes the installation prerequisites and procedures. Read this if you need to install Red Hat Enterprise Virtualization. The installation of
16
Chapt er 1 . Int roduct ion
host physical machines, Manager and storage are covered in this guide. You will need to refer to the Red Hat Enterprise Virtualization Administration Guide to configure the system before you can start using the platform.
Red Hat Enterprise Virtualization — Manager Release Notes contain release specific information for
Red Hat Enterprise Virtualization Managers.
Red Hat Enterprise Virtualization — Power User Portal Guide describes how power users can create and manage virtual machines from the Red Hat Enterprise Virtualization User Portal.
Red Hat Enterprise Virtualization — Quick Start Guide provides quick and simple instructions for first time users to set up a basic Red Hat Enterprise Virtualization environment.
Red Hat Enterprise Virtualization — Technical Notes describe the changes made between the current release and the previous one.
Red Hat Enterprise Virtualization — Technical Reference Guide describes the technical architecture of
Red Hat Enterprise Virtualization and its interactions with existing infrastructure.
Red Hat Enterprise Virtualization — User Portal Guide describes how users of the Red Hat Enterprise
Virtualization system can access and use virtual desktops from the User Portal.
Note
All of the guides for these products are available at the Red Hat Customer Portal: https://access.redhat.com/site/documentation/ .
17
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 2. Server best practices
The following tasks and tips can assist you with increasing the performance of your Red Hat
Enterprise Linux host. Additional tips can be found in the Red Hat Enterprise Linux Virtualization Tuning
and Optimization Guide
Run SELinux in enforcing mode. Set SELinux to run in enforcing mode with the setenfo rce command.
# setenforce 1
Remove or disable any unnecessary services such as Auto FS, NFS, FT P , HT T P , NIS, tel netd ,
send mai l and so on.
Only add the minimum number of user accounts needed for platform management on the server and remove unnecessary user accounts.
Avoid running any unessential applications on your host. Running applications on the host may impact virtual machine performance and can affect server stability. Any application which may crash the server will also cause all virtual machines on the server to go down.
Use a central location for virtual machine installations and images. Virtual machine images should be stored under /var/l i b/l i bvi rt/i mag es/. If you are using a different directory for your virtual machine images make sure you add the directory to your SELinux policy and relabel it before starting the installation. Use of shareable, network storage in a central location is highly recommended.
18
Chapt er 3. Securit y for virt ualizat ion
Chapter 3. Security for virtualization
When deploying virtualization technologies, you must ensure that the host physical machine and its operating system cannot be compromised. In this case the host is a Red Hat Enterprise Linux system that manages the system, devices, memory and networks as well as all guest virtual machines. If the host physical machine is insecure, all guest virtual machines in the system are vulnerable. There are several ways to enhance security on systems using virtualization. You or your organization should create a Deployment Plan. This plan needs to contain the following:
Operating specifications
Specifies which services are needed on your guest virtual machines
Specifies the host physical servers as well as what support is required for these services
Here are a few security issues to consider while developing a deployment plan:
Run only necessary services on host physical machines. The fewer processes and services running on the host physical machine, the higher the level of security and performance.
Enable SELinux on the hypervisor. Read Section 3.2, “ SELinux and virtualization”
for more information on using SELinux and virtualization. Additional security tips are located in the Red
Hat Enterprise Linux Virtualization Security Guide
Use a firewall to restrict traffic to the host physical machine. You can setup a firewall with defaultreject rules that will help secure the host physical machine from attacks. It is also important to limit network-facing services.
Do not allow normal users to access the host operating system. If the host operating system is privileged, granting access to unprivileged accounts may compromise the level of security.
3.1. St orage securit y issues
Keeping in mind that any user with administrative security privileges for guest virtual machines can potentially change the partitions in the host physical machine, it is imperative that only actual system administrators are granted this level of security. In addition, the following should be considered:
The host physical machine should not use disk labels to identify file systems in the fstab file, the
i ni trd file which are accessed by the command line. If less privileged users, especially users of guest virtual machines have write access to whole partitions or LVM volumes, then they can be accidentally deleted and this mistake will impact all other guest virtual machines that are using the same storage.
Users of guest virtual machines should not be given write access to entire disks or block devices
(for example, /d ev/sd b). To avoid this, use partitions such as /d ev/sd b1 or LVM volumes.
3.2. SELinux and virt ualizat ion
Security Enhanced Linux was developed by the NSA with assistance from the Linux community to provide stronger security for Linux. SELinux limits an attacker's abilities and works to prevent many common security exploits such as buffer overflow attacks and privilege escalation. It is because of these benefits that all Red Hat Enterprise Linux systems should run with SELinux enabled and in enforcing mode.
Pro ced u re 3.1. Creat in g an d mo u n t in g a lo g ical vo lu me o n a g u est virt u al mach in e wit h
SELin u x en ab led
19
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
1. Create a logical volume. This example creates a 5 gigabyte logical volume named
NewVolumeName on the volume group named volumegroup. This example also assumes that there is enough disk space. You may have to create additional storage on a network
device and give the guest access to it. Refer to Chapter 14, Volumes for more information.
# lvcreate -n NewVolumeName -L 5G volumegroup
2. Format the NewVolumeName logical volume with a file system that supports extended attributes, such as ext3.
# mke2fs -j /dev/volumegroup/NewVolumeName
3. Create a new directory for mounting the new logical volume. This directory can be anywhere on your file system. It is advised not to put it in important system directories (/etc, /var,
/sys) or in home directories (/ho me or /ro o t). This example uses a directory called
/vi rtsto rag e
# mkdir /virtstorage
4. Mount the logical volume.
# mount /dev/volumegroup/NewVolumeName /virtstorage
5. Set the SELinux type for the folder you just created.
# semanage fcontext -a -t virt_image_t "/virtstorage(/.*)?"
If the targeted policy is used (targeted is the default policy) the command appends a line to the /etc/sel i nux/targ eted /co ntexts/fi l es/fi l e_co ntexts. l o cal file which makes the change persistent. The appended line may resemble this:
/virtstorage(/.*)? system_u:object_r:virt_image_t:s0
6. Run the command to change the type of the mount point (/vi rtsto rag e) and all files under it to vi rt_i mag e_t (the resto reco n and setfi l es commands read the files in
/etc/sel i nux/targ eted /co ntexts/fi l es/).
# restorecon -R -v /virtstorage
20
Chapt er 3. Securit y for virt ualizat ion
Note
Create a new file (using the to uch command) on the file system.
# touch /virtstorage/newfile
Verify the file has been relabeled using the following command:
# sudo ls -Z /virtstorage
-rw-------. root root system_u:object_r:virt_image_t:s0 newfile
The output shows that the new file has the correct attribute, vi rt_i mag e_t.
3.3. SELinux
This section contains topics to consider when using SELinux with your virtualization deployment.
When you deploy system changes or add devices, you must update your SELinux policy accordingly. To configure an LVM volume for a guest virtual machine, you must modify the SELinux context for the respective underlying block device and volume group. Make sure that you have installed the po l i cyco reuti l s-pytho n package (yum i nstal l po l i cyco reuti l zs-
pytho n) before running the command.
# semanage fcontext -a -t virt_image_t -f -b /dev/sda2
# restorecon /dev/sda2
KVM an d SELin u x
The following table shows the SELinux Booleans which affect KVM when launched by libvirt.
KVM SELin u x Bo o lean s
SELin u x Bo o lean
virt_use_comm virt_use_fusefs virt_use_nfs virt_use_samba virt_use_sanlock virt_use_sysfs virt_use_xserver virt_use_usb
Descrip t io n
Allow virt to use serial/parallel communication ports.
Allow virt to read fuse files.
Allow virt to manage NFS files.
Allow virt to manage CIFS files.
Allow sanlock to manage virt lib files.
Allow virt to manage device configuration (PCI).
Allow virtual machine to interact with the xserver.
Allow virt to use USB devices.
3.4 . Virt ualizat ion firewall informat ion
Various ports are used for communication between guest virtual machines and cooresponding management utilities.
21
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
Any network service on a guest virtual machine must have the applicable ports open on the guest virtual machine to allow external access. If a network service on a guest virtual machine is firewalled it will be inaccessible. Always verify the guest virtual machine's network configuration first.
ICMP requests must be accepted. ICMP packets are used for network testing. You cannot ping guest virtual machines if the ICMP packets are blocked.
Port 22 should be open for SSH access and the initial installation.
Ports 80 or 443 (depending on the security settings on the RHEV Manager) are used by the vdsmreg service to communicate information about the host physical machine.
Ports 5634 to 6166 are used for guest virtual machine console access with the SPICE protocol.
Ports 49152 to 49216 are used for migrations with KVM. Migration may use any port in this range depending on the number of concurrent migrations occurring.
Enabling IP forwarding (net. i pv4 . i p_fo rward = 1) is also required for shared bridges and the default bridge. Note that installing libvirt enables this variable so it will be enabled when the virtualization packages are installed unless it was manually disabled.
Note
Note that enabling IP forwarding is n o t required for physical bridge devices. When a guest virtual machine is connected through a physical bridge, traffic only operates at a level that does not require IP configuration such as IP forwarding.
22
Chapt er 4 . sVirt
Chapter 4. sVirt
sVirt is a technology included in Red Hat Enterprise Linux 6 that integrates SELinux and virtualization. sVirt applies Mandatory Access Control (MAC) to improve security when using guest virtual machines. This integrated technology improves security and hardens the system against bugs in the hypervisor. It is particulary helpful in preventing attacks on the host physical machine or on another guest virtual machine.
This chapter describes how sVirt integrates with virtualization technologies in Red Hat Enterprise
Linux 6.
No n - virt u aliz ed en viro n men t s
In a non-virtualized environment, host physical machines are separated from each other physically and each host physical machine has a self-contained environment, consisting of services such as a web server, or a DNS server. These services communicate directly to their own user space, host physical machine's kernel and physical hardware, offering their services directly to the network. The following image represents a non-virtualized environment:
User Space - memory area where all user mode applications and some drivers execute.
Web App (web application server) - delivers web content that can be accessed through the a browser.
Host Kernel - is strictly reserved for running the host physical machine's privileged kernel, kernel extensions, and most device drivers.
DNS Server - stores DNS records allowing users to access web pages using logical names instead of IP addresses.
Virt u aliz ed en viro n men t s
In a virtualized environment, several virtual operating systems can run on a single kernel residing on a host physical machine. The following image represents a virtualized environment:
23
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
4 .1. Securit y and Virt ualizat ion
When services are not virtualized, machines are physically separated. Any exploit is usually contained to the affected machine, with the obvious exception of network attacks. When services are grouped together in a virtualized environment, extra vulnerabilities emerge in the system. If there is a security flaw in the hypervisor that can be exploited by a guest virtual machine, this guest virtual machine may be able to not only attack the host physical machine, but also other guest virtual machines running on that host physical machine. These attacks can extend beyond the guest virtual machine and could expose other guest virtual machines to an attack as well.
sVirt is an effort to isolate guest virtual machines and limit their ability to launch further attacks if exploited. This is demonstrated in the following image, where an attack can not break out of the guest virtual machine and invade other guest virtual machines:
SELinux introduces a pluggable security framework for virtualized instances in its implementation of
Mandatory Access Control (MAC). The sVirt framework allows guest virtual machines and their resources to be uniquely labeled. Once labeled, rules can be applied which can reject access between different guest virtual machines.
24
Chapt er 4 . sVirt
4 .2. sVirt labeling
Like other services under the protection of SELinux, sVirt uses process-based mechanisms and restrictions to provide an extra layer of security over guest virtual machines. Under typical use, you should not even notice that sVirt is working in the background. This section describes the labeling features of sVirt.
As shown in the following output, when using sVirt, each virtualized guest virtual machine process is labeled and runs with a dynamically generated level. Each process is isolated from other VMs with different levels:
# ps -eZ | grep qemu system_u:system_r:svirt_t:s0:c87,c520 27950 ? 00:00:17 qemu-kvm
The actual disk images are automatically labeled to match the processes, as shown in the following output:
# ls -lZ /var/lib/libvirt/images/*
system_u:object_r:svirt_image_t:s0:c87,c520 image1
The following table outlines the different context labels that can be assigned when using sVirt:
T ab le 4 .1. sVirt co n t ext lab els
SELin u x Co n t ext
system_u:system_r:svirt_t:MCS1 system_u:object_r:svirt_image_t:MCS1 system_u:object_r:svirt_image_t:s0
T yp e / Descrip t io n
Guest virtual machine processes. MCS1 is a random MCS field. Approximately 500,000 labels are supported.
Guest virtual machine images. Only svirt_t processes with the same MCS fields can read/write these images.
Guest virtual machine shared read/write content.
All svirt_t processes can write to the svirt_image_t:s0 files.
It is also possible to perform static labeling when using sVirt. Static labels allow the administrator to select a specific label, including the MCS/MLS field, for a guest virtual machine. Administrators who run statically-labeled virtualized guest virtual machines are responsible for setting the correct label on the image files. The guest virtual machine will always be started with that label, and the sVirt system will never modify the label of a statically-labeled virtual machine's content. This allows the sVirt component to run in an MLS environment. You can also run multiple guest virtual machines with different sensitivity levels on a system, depending on your requirements.
25
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 5. KVM live migration
This chapter covers migrating guest virtual machines running on one host physical machine to another. In both instances, the host physical machines are running the KVM hypervisor.
Migration describes the process of moving a guest virtual machine from one host physical machine to another. This is possible because guest virtual machines are running in a virtualized environment instead of directly on the hardware. Migration is useful for:
Load balancing - guest virtual machines can be moved to host physical machines with lower usage when their host physical machine becomes overloaded, or another host physical machine is under-utilized.
Hardware independence - when we need to upgrade, add, or remove hardware devices on the host physical machine, we can safely relocate guest virtual machines to other host physical machines. This means that guest virtual machines do not experience any downtime for hardware improvements.
Energy saving - guest virtual machines can be redistributed to other host physical machines and can thus be powered off to save energy and cut costs in low usage periods.
Geographic migration - guest virtual machines can be moved to another location for lower latency or in serious circumstances.
Migration works by sending the state of the guest virtual machine's memory and any virtualized devices to a destination host physical machine. It is recommended to use shared, networked storage to store the guest virtual machine's images to be migrated. It is also recommended to use libvirtmanaged storage pools for shared storage when migrating virtual machines.
Migrations can be performed live or not.
In a live migration, the guest virtual machine continues to run on the source host physical machine while its memory pages are transferred, in order, to the destination host physical machine. During migration, KVM monitors the source for any changes in pages it has already transferred, and begins to transfer these changes when all of the initial pages have been transferred. KVM also estimates transfer speed during migration, so when the remaining amount of data to transfer will take a certain configurable period of time (10ms by default), KVM suspends the original guest virtual machine, transfers the remaining data, and resumes the same guest virtual machine on the destination host physical machine.
A migration that is not performed live, suspends the guest virtual machine, then moves an image of the guest virtual machine's memory to the destination host physical machine. The guest virtual machine is then resumed on the destination host physical machine and the memory the guest virtual machine used on the source host physical machine is freed. The time it takes to complete such a migration depends on network bandwidth and latency. If the network is experiencing heavy use or low bandwidth, the migration will take much longer.
If the original guest virtual machine modifies pages faster than KVM can transfer them to the destination host physical machine, offline migration must be used, as live migration would never complete.
5.1. Live migrat ion requirement s
Migrating guest virtual machines requires the following:
26
Chapt er 5. KVM live migrat ion
Mig rat io n req u iremen t s
A guest virtual machine installed on shared storage using one of the following protocols:
Fibre Channel-based LUNs iSCSI
FCoE
NFS
GFS2
SCSI RDMA protocols (SCSI RCP): the block export protocol used in Infiniband and 10GbE iWARP adapters
The migration platforms and versions should be checked against table Table 5.1, “ Live Migration
Compatibility” . It should also be noted that Red Hat Enterprise Linux 6 supports live migration of
guest virtual machines using raw and qcow2 images on shared storage.
Both systems must have the appropriate TCP/IP ports open. In cases where a firewall is used refer
to Section 3.4, “ Virtualization firewall information”
for detailed port information.
A separate system exporting the shared storage medium. Storage should not reside on either of the two host physical machines being used for migration.
Shared storage must mount at the same location on source and destination systems. The mounted directory names must be identical. Although it is possible to keep the images using different paths, it is not recommended. Note that, if you are intending to use virt-manager to perform the migration, the path names must be identical. If however you intend to use virsh to perform the migration, different network configurations and mount directories can be used with the help of --xml option or pre-hooks when doing migrations. Even without shared storage, migration can still succeed with the option --co py-sto rag e-al l (deprecated). For more information on preho o ks, refer to libvirt.org
, and for more information on the XML option, refer to
Chapter 21, Manipulating the domain xml
.
When migration is attempted on an existing guest virtual machine in a public bridge+tap network, the source and destination host physical machines must be located in the same network.
Otherwise, the guest virtual machine network will not operate after migration.
Red Hat Enterprise Linux 5 and 6 default cache mode of KVM guest virtual machines is set to
none, which prevents inconsistent disk states. When cache=none is used for a guest disk image, all of the files will be opened with O_DIRECT. There will be no caching on the KVM host physical machine and there will be caching only on the guest virtual machine. Hence, there will not be any data inconsistency problems when cache=none is used. It is this default cache mode of KVM guest virtial machines that makes live migration of guest virtual machines possible, so it should not be changed.
Make sure that the l i bvi rtd service is enabled (# chkco nfi g l i bvi rtd o n) and running (#
servi ce l i bvi rtd start). It is also important to note that the ability to migrate effectively is dependent on the parameter settings in the /etc/l i bvi rt/l i bvi rtd . co nf configuration file.
Pro ced u re 5.1. Co n f ig u rin g lib virt d .co n f
1. Opening the l i bvi rtd . co nf requires running the command as root:
# vim /etc/libvirt/libvirtd.conf
27
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
2. Change the parameters as needed and save the file.
3. Restart the l i bvi rtd service:
# service libvirtd restart
5.2. Live migrat ion and Red Hat Ent erprise Linux version compat ibilit y
Live Migration is supported as shown in table Table 5.1, “ Live Migration Compatibility”
:
T ab le 5.1. Live Mig rat io n Co mp at ib ilit y
Release T yp e Examp le Mig rat io n
Met h o d
Forward
Forward
Major release
Minor release
5.x → 6.y
5.x → 5.y (y>x, x>=4)
Forward
Backward
Backward
Backward
Minor release
Major release
Minor release
Minor release
6.x → 6.y (y>x, x>=0)
6.x → 5.y
5.x → 5.y
(x>y,y>=4)
6.x → 6.y (x>y, y>=0)
Live Mig rat io n
Su p p o rt
Not supported
Fully supported
Fully supported
Not supported
Supported
Supported
No t es
Any issues should be reported
Any issues should be reported
Refer to
Troubleshooting problems with migration for
known issues
Refer to
Troubleshooting problems with migration for
known issues
T ro u b lesh o o t in g p ro b lems wit h mig rat io n
Issu es wit h SPICE — It has been found that SPICE has an incompatible change when migrating from 6.0 → 6.1. In such cases, the client may disconnect and then reconnect, causing a temporary loss of audio and video. This is only temporary and all services will resume.
Issu es wit h USB — Red Hat Enterprise Linux 6.2 added USB functionality which included migration support, but not without certain caveats which reset USB devices and caused any application running over the device to abort. This problem was fixed in Red Hat Enterprise Linux
6.4, and should not occur in future versions. To prevent this from happening in a version prior to
6.4, abstain from migrating while USB devices are in use.
Issu es wit h t h e mig rat io n p ro t o co l — If backward migration ends with "unknown section error", repeating the migration process can repair the issue as it may be a transient error. If not, please report the problem.
Co n f ig u rin g n et wo rk st o rag e
Configure shared storage and install a guest virtual machine on the shared storage.
28
Chapt er 5. KVM live migrat ion
5.3. Shared st orage example: NFS for a simple migrat ion
Important
This example uses NFS to share guest virtual machine images with other KVM host physical machines. Although not practical for large installations, it is presented to demonstrate migration techniques only. Do not use this example for migrating or running more than a few guest virtual machines. In addition, it is required that the sync parameter is enabled. This is required for proper export of the NFS storage.
Also note, that the instructions provided herein are not meant to replace the detailed instructions found in Red Hat Linux Storage Administration Guide. Refer to this guide for information on configuring
NFS, opening IP tables, and configuring the firewall.
Make sure that NFS filelocking is not used as it is not supported in KVM.
1. Exp o rt yo u r lib virt imag e d irect o ry
Migration requires storage to reside on a system that is separate to the migration target systems. On this separate system, export the storage by adding the default image directory to the /etc/expo rts file:
/var/lib/libvirt/images *.example.com(rw,no_root_squash,sync)
Change the hostname parameter as required for your environment.
2. St art NFS a. Make sure that the ports for NFS in i ptabl es (2049, for example) are opened and add NFS to the /etc/ho sts. al l o w file.
b. Start the NFS service:
# service nfs start
3. Mo u n t t h e sh ared st o rag e o n t h e d est in at io n
On the migration destination system, mount the /var/l i b/l i bvi rt/i mag es directory:
# mount storage_host:/var/lib/libvirt/images
/var/lib/libvirt/images
29
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Warning
Whichever directory is chosen for the guest virtual machine must be exactly the same as that on the host physical machine. This applies to all types of shared storage. The directory must be the same or the migration with virt-manager will fail.
5.4 . Live KVM migrat ion wit h virsh
A guest virtual machine can be migrated to another host physical machine with the vi rsh command.
The mi g rate command accepts parameters in the following format:
# virsh migrate --live GuestName DestinationURL
Note that the --live option may be eliminated when live migration is not desired. Additional options are
listed in Section 5.4.2, “ Additional options for the virsh migrate command” .
The GuestName parameter represents the name of the guest virtual machine which you want to migrate.
The DestinationURL parameter is the connection URL of the destination host physical machine.
The destination system must run the same version of Red Hat Enterprise Linux, be using the same hypervisor and have l i bvi rt running.
Note
The DestinationURL parameter for normal migration and peer2peer migration has different semantics: normal migration: the DestinationURL is the URL of the target host physical machine as seen from the source guest virtual machine.
peer2peer migration: DestinationURL is the URL of the target host physical machine as seen from the source host physical machine.
Once the command is entered, you will be prompted for the root password of the destination system.
Important
An entry for the destination host physical machine, in the /etc/ho sts file on the source server is required for migration to succeed. Enter the IP address and hostname for the destination host physical machine in this file as shown in the following example, substituting your destination host physical machine's IP address and hostname:
10.0.0.20 host2.example.com
Examp le: live mig rat io n wit h virsh
30
Chapt er 5. KVM live migrat ion
This example migrates from ho st1. exampl e. co m to ho st2. exampl e. co m. Change the host physical machine names for your environment. This example migrates a virtual machine named
g uest1-rhel 6 -6 4 .
This example assumes you have fully configured shared storage and meet all the prerequisites
(listed here: Migration requirements
).
1.
Verif y t h e g u est virt u al mach in e is ru n n in g
From the source system, ho st1. exampl e. co m, verify g uest1-rhel 6 -6 4 is running:
[root@ host1 ~]# virsh list
Id Name State
----------------------------------
10 guest1-rhel6-64 running
2.
Mig rat e t h e g u est virt u al mach in e
Execute the following command to live migrate the guest virtual machine to the destination,
ho st2. exampl e. co m. Append /system to the end of the destination URL to tell libvirt that you need full access.
# virsh migrate --live guest1-rhel6-64
qemu+ssh://host2.example.com/system
3.
Once the command is entered you will be prompted for the root password of the destination system.
4.
Wait
The migration may take some time depending on load and the size of the guest virtual machine. vi rsh only reports errors. The guest virtual machine continues to run on the source host physical machine until fully migrated.
Verif y t h e g u est virt u al mach in e h as arrived at t h e d est in at io n h o st
From the destination system, ho st2. exampl e. co m, verify g uest1-rhel 6 -6 4 is running:
[root@ host2 ~]# virsh list
Id Name State
----------------------------------
10 guest1-rhel6-64 running
The live migration is now complete.
31
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
libvirt supports a variety of networking methods including TLS/SSL, UNIX sockets, SSH, and
unencrypted TCP. Refer to Chapter 6, Remote management of guests
for more information on using other methods.
Note
Non-running guest virtual machines cannot be migrated with the vi rsh mi g rate command.
To migrate a non-running guest virtual machine, the following script should be used: virsh dumpxml Guest1 > Guest1.xml
virsh -c qemu+ssh://<target-system-FQDN> define Guest1.xml
virsh undefine Guest1
5.4 .1. Addit ional t ips for migrat ion wit h virsh
It is possible to perform multiple, concurrent live migrations where each migration runs in a separate command shell. However, this should be done with caution and should involve careful calculations as each migration instance uses one MAX_CLIENT from each side (source and target). As the default setting is 20, there is enough to run 10 instances without changing the settings. Should you need to
change the settings, refer to the procedure Procedure 5.1, “ Configuring libvirtd.conf”
.
1. Open the libvirtd.conf file as described in Procedure 5.1, “ Configuring libvirtd.conf”
.
2. Look for the Processing controls section.
#################################################################
#
# Processing controls
#
# The maximum number of concurrent client connections to allow
# over all sockets combined.
#max_clients = 20
# The minimum limit sets the number of workers to start up
# initially. If the number of active clients exceeds this,
# then more threads are spawned, upto max_workers limit.
# Typically you'd want max_workers to equal maximum number
# of clients allowed
#min_workers = 5
#max_workers = 20
# The number of priority workers. If all workers from above
# pool will stuck, some calls marked as high priority
# (notably domainDestroy) can be executed in this pool.
#prio_workers = 5
32
Chapt er 5. KVM live migrat ion
# Total global limit on concurrent RPC calls. Should be
# at least as large as max_workers. Beyond this, RPC requests
# will be read into memory and queued. This directly impact
# memory usage, currently each request requires 256 KB of
# memory. So by default upto 5 MB of memory is used
#
# XXX this isn't actually enforced yet, only the per-client
# limit is used so far
#max_requests = 20
# Limit on concurrent requests from a single client
# connection. To avoid one client monopolizing the server
# this should be a small fraction of the global max_requests
# and max_workers parameter
#max_client_requests = 5
#################################################################
3. Change the max_clients and max_workers parameters settings. It is recommended that the number be the same in both parameters. The max_clients will use 2 clients per migration (one per side) and max_workers will use 1 worker on the source and 0 workers on the destination during the perform phase and 1 worker on the destination during the finish phase.
Important
The max_clients and max_workers parameters settings are effected by all guest virtual machine connections to the libvirtd service. This means that any user that is using the same guest virtual machine and is performing a migration at the same time will also beholden to the limits set in the the max_clients and max_workers parameters settings. This is why the maximum value needs to be considered carefully before performing a concurrent live migration.
4. Save the file and restart the service.
Note
There may be cases where a migration connection drops because there are too many ssh sessions that have been started, but not yet authenticated. By default, sshd allows only 10 sessions to be in a "pre-authenticated state" at any time. This setting is controlled by the MaxStartups parameter in the sshd configuration file (located here:
/etc/ssh/sshd _co nfi g ), which may require some adjustment. Adjusting this parameter should be done with caution as the limitation is put in place to prevent DoS attacks (and over-use of resources in general). Setting this value too high will negate its purpose. To change this parameter, edit the file /etc/ssh/sshd _co nfi g , remove the # from the beginning of the MaxStartups line, and change the 10 (default value) to a higher number. Remember to save the file and restart the sshd service. For more information, refer to the sshd _co nfi g MAN page.
5.4 .2. Addit ional opt ions for t he virsh migrat e command
33
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
In addition to --live, virsh migrate accepts the following options:
--direct - used for direct migration
--p2p - used for peer-2-peer migration
--tunnelled - used for tunneled migration
--persistent - leaves the domain in a persistent state on the destination host physical machine
--undefinesource - removes the guest virtual machine on the source host physical machine
--suspend - leaves the domain in a paused state on the destination host physical machine
--copy-storage-all (deprecated) - indicates migration with non-shared storage with full disk copy.
--copy-storage-inc (deprecated) - indicates migration with non-shared storage with incremental copy (same base image shared between source and destination). In both cases the disk images have to exist on the destination host physical machine, the --copy-storage-*. (deprecated) options only tell libvirt to transfer data from the images on source host physical machine to the images found at the same place on the destination host physical machine. For more information see Does kvm support live migration with non-shared storage?
on the customer portal.
--change-protection - enforces that no incompatible configuration changes will be made to the domain while the migration is underway; this flag is implicitly enabled when supported by the hypervisor, but can be explicitly used to reject the migration if the hypervisor lacks change protection support.
--unsafe - forces the migration to occur, ignoring all safety procedures.
--verbose - displays the progress of migration as it is occurring
[--abort-on-error] - cancels the migration if a soft error (such as an I/O error) happens during the migration process.
[--migrateuri] - the migration URI which is usually omitted.
[--domain <string>]- domain name, id or uuid
[--desturi <string>] - connection URI of the destination host physical machine as seen from the client(normal migration) or source(p2p migration)
[--migrateuri] - migration URI, usually can be omitted
--timeout <seconds> - forces a guest virtual machine to suspend when the live migration counter exceeds N seconds. It can only be used with a live migration. Once the timeout is initiated, the migration continues on the suspended guest virtual machine.
dname - is used for renaming the domain to new name during migration, which also usually can be omitted
[--dname] <string> - changes the name of the guest virtual machine to a new name during migration
(if supported)
--xml - the filename indicated can be used to supply an alternative XML file for use on the destination to supply a larger set of changes to any host-specific portions of the domain XML, such as accounting for naming differences between source and destination in accessing underlying storage. This option is usually omitted.
Refer to the virsh man page for more information.
34
Chapt er 5. KVM live migrat ion
5.5. Migrat ing wit h virt -manager
This section covers migrating a KVM guest virtual machine with vi rt-manag er from one host physical machine to another.
1. O p en virt - man ag er
Open vi rt-manag er. Choose Ap p licat io n s → Syst em T o o ls → Virt u al Mach in e
Man ag er from the main menu bar to launch vi rt-manag er.
Fig u re 5.1. Virt - Man ag er main men u
2. Co n n ect t o t h e t arg et h o st p h ysical mach in e
Connect to the target host physical machine by clicking on the File menu, then click Ad d
Co n n ect io n .
35
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 5.2. O p en Ad d Co n n ect io n win d o w
3. Ad d co n n ect io n
The Ad d C o nnecti o n window appears.
36
Fig u re 5.3. Ad d in g a co n n ect io n t o t h e t arg et h o st p h ysical mach in e
Enter the following details:
Hypervi so r: Select Q EMU/KVM.
Chapt er 5. KVM live migrat ion
Metho d : Select the connection method.
Username: Enter the username for the remote host physical machine.
Ho stname: Enter the hostname for the remote host physical machine.
Click the C o nnect button. An SSH connection is used in this example, so the specified user's password must be entered in the next step.
Fig u re 5.4 . En t er p asswo rd
4. Mig rat e g u est virt u al mach in es
Open the list of guests inside the source host physical machine (click the small triangle on the left of the host name) and right click on the guest that is to be migrated (g u est 1- rh el6 -
6 4 in this example) and click Mig rat e.
37
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
38
Fig u re 5.5. Ch o o sin g t h e g u est t o b e mig rat ed
In the New Ho st field, use the drop-down list to select the host physical machine you wish to migrate the guest virtual machine to and click Mig rat e.
Chapt er 5. KVM live migrat ion
Fig u re 5.6 . Ch o o sin g t h e d est in at io n h o st p h ysical mach in e an d st art in g t h e mig rat io n p ro cess
A progress window will appear.
Fig u re 5.7. Pro g ress win d o w
39
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
virt - man ag er now displays the newly migrated guest virtual machine running in the destination host. The guest virtual machine that was running in the source host physical machine is now listed inthe Shutoff state.
4 0
Fig u re 5.8. Mig rat ed g u est virt u al mach in e ru n n in g in t h e d est in at io n h o st p h ysical mach in e
5. O p t io n al - View t h e st o rag e d et ails f o r t h e h o st p h ysical mach in e
In the Ed it menu, click Co n n ect io n Det ails, the Connection Details window appears.
Click the Sto rag e tab. The iSCSI target details for the destination host physical machine is shown. Note that the migrated guest virtual machine is listed as using the storage
Chapt er 5. KVM live migrat ion
Fig u re 5.9 . St o rag e d et ails
This host was defined by the following XML configuration:
<pool type= 'iscsi' >
<name> iscsirhel6guest </name>
<source>
<host name= 'virtlab22.example.com.' />
<device path= 'iqn.2001-05.com.iscsivendor:0-8a0906fbab74a06-a700000017a4cc89-rhevh' />
</source>
<target>
<path> /dev/disk/by-path </path>
</target>
</pool>
...
Fig u re 5.10. XML co n f ig u rat io n f o r t h e d est in at io n h o st p h ysical mach in e
4 1
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 6. Remote management of guests
This section explains how to remotely manage your guests using ssh or TLS and SSL. More information on SSH can be found in the Red Hat Enterprise Linux Deployment Guide
6.1. Remot e management wit h SSH
The ssh package provides an encrypted network protocol which can securely send management functions to remote virtualization servers. The method described uses the l i bvi rt management connection securely tunneled over an SSH connection to manage the remote machines. All the authentication is done using SSH public key cryptography and passwords or passphrases gathered by your local SSH agent. In addition the VNC console for each guest is tunneled over SSH.
Be aware of the issues with using SSH for remotely managing your virtual machines, including: you require root log in access to the remote machine for managing virtual machines, the initial connection setup process may be slow, there is no standard or trivial way to revoke a user's key on all hosts or guests, and ssh does not scale well with larger numbers of remote machines.
Note
Red Hat Enterprise Virtualization enables remote management of large numbers of virtual machines. Refer to the Red Hat Enterprise Virtualization documentation for further details.
The following packages are required for ssh access:
openssh openssh-askpass openssh-clients openssh-server
Co n f ig u rin g p asswo rd less o r p asswo rd man ag ed SSH access f o r vi rt-manag er
The following instructions assume you are starting from scratch and do not already have SSH keys set up. If you have SSH keys set up and copied to the other systems you can skip this procedure.
4 2
Chapt er 6 . Remot e management of guest s
Important
SSH keys are user dependent and may only be used by their owners. A key's owner is the one who generated it. Keys may not be shared.
vi rt-manag er must be run by the user who owns the keys to connect to the remote host. That means, if the remote systems are managed by a non-root user vi rt-manag er must be run in unprivileged mode. If the remote systems are managed by the local root user then the SSH keys must be owned and created by root.
You cannot manage the local host as an unprivileged user with vi rt-manag er.
1. O p t io n al: Ch an g in g u ser
Change user, if required. This example uses the local root user for remotely managing the other hosts and the local host.
$ su -
2. G en erat in g t h e SSH key p air
Generate a public key pair on the machine vi rt-manag er is used. This example uses the default key location, in the ~ /. ssh/ directory.
# ssh-keyg en -t rsa
3. Co p yin g t h e keys t o t h e remo t e h o st s
Remote login without a password, or with a passphrase, requires an SSH key to be distributed to the systems being managed. Use the ssh-co py-i d command to copy the key to root user at the system address provided (in the example, [email protected]).
# ssh-co py-i d -i ~ /. ssh/i d _rsa. pub ro o t@ ho st2. exampl e. co m root@ host2.example.com's password:
Now try logging into the machine, with the ssh ro o t@ ho st2. exampl e. co m command and check in the . ssh/autho ri zed _keys file to make sure unexpected keys have not been added.
Repeat for other systems, as required.
4. O p t io n al: Ad d t h e p assp h rase t o t h e ssh - ag en t
The instructions below describe how to add a passphrase to an existing ssh-agent. It will fail to run if the ssh-agent is not running. To avoid errors or conflicts make sure that your SSH parameters are set correctly. Refer to the Red Hat Enterprise Linux Deployment Guide for more information.
Add the passphrase for the SSH key to the ssh-ag ent, if required. On the local host, use the following command to add the passphrase (if there was one) to enable password-less login.
# ssh-ad d ~ /. ssh/i d _rsa. pub
4 3
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The SSH key is added to the remote system.
T h e l i bvi rt d aemo n ( l i bvi rtd )
The l i bvi rt daemon provides an interface for managing virtual machines. You must have the
l i bvi rtd daemon installed and running on every remote host that needs managing.
$ ssh ro o t@ somehost
# chkco nfi g l i bvi rtd o n
# servi ce l i bvi rtd start
After l i bvi rtd and SSH are configured you should be able to remotely access and manage your virtual machines. You should also be able to access your guests with VNC at this point.
Accessin g remo t e h o st s wit h virt - man ag er
Remote hosts can be managed with the virt-manager GUI tool. SSH keys must belong to the user executing virt-manager for password-less login to work.
1. Start virt-manager.
2. Open the File->Ad d Co n n ect io n menu.
Fig u re 6 .1. Ad d co n n ect io n men u
3. Use the drop down menu to select hypervisor type, and click the Co n n ect t o remo t e h o st check box to open the Connection Met h o d (in this case Remote tunnel over SSH), and enter the desired User n ame and Ho st n ame, then click Co n n ect .
6.2. Remot e management over T LS and SSL
4 4
Chapt er 6 . Remot e management of guest s
You can manage virtual machines using TLS and SSL. TLS and SSL provides greater scalability but is more complicated than ssh (refer to
Section 6.1, “ Remote management with SSH” ). TLS and SSL is
the same technology used by web browsers for secure connections. The l i bvi rt management connection opens a TCP port for incoming connections, which is securely encrypted and authenticated based on x509 certificates. The procedures that follow provide instructions on creating and deploying authentication certificates for TLS and SSL management.
Pro ced u re 6 .1. Creat in g a cert if icat e au t h o rit y ( CA) key f o r T LS man ag emen t
1. Before you begin, confirm that the certto o l utility is installed. If not:
# yum i nstal l g nutl s-uti l s
2. Generate a private key, using the following command:
# certto o l --g enerate-pri vkey > cakey. pem
3. Once the key generates, the next step is to create a signature file so the key can be selfsigned. To do this, create a file with signature details and name it ca. i nfo . This file should contain the following:
# vi m ca. i nfo cn = Name of your organization ca cert_signing_key
4. Generate the self-signed key with the following command:
# certto o l --g enerate-sel f-si g ned --l o ad -pri vkey cakey. pem --
templ ate ca. i nfo --o utfi l e cacert. pem
Once the file generates, the ca.info file may be deleted using the rm command. The file that results from the generation process is named cacert. pem. This file is the public key
(certificate). The loaded file cakey. pem is the private key. This file should not be kept in a shared space. Keep this key private.
5. Install the cacert. pem Certificate Authority Certificate file on all clients and servers in the
/etc/pki /C A/cacert. pem directory to let them know that the certificate issued by your CA can be trusted. To view the contents of this file, run:
# certto o l -i --i nfi l e cacert. pem
This is all that is required to set up your CA. Keep the CA's private key safe as you will need it in order to issue certificates for your clients and servers.
Pro ced u re 6 .2. Issu in g a server cert if icat e
This procedure demonstrates how to issue a certificate with the X.509 CommonName (CN)field set to the hostname of the server. The CN must match the hostname which clients will be using to connect to the server. In this example, clients will be connecting to the server using the URI:
q emu: //myco mmo nname/system, so the CN field should be identical, ie mycommoname.
1. Create a private key for the server.
4 5
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# certto o l --g enerate-pri vkey > serverkey. pem
2. Generate a signature for the CA's private key by first creating a template file called
server. i nfo . Make sure that the CN is set to be the same as the server's hostname: organization = Name of your organization cn = mycommonname tls_www_server encryption_key signing_key
3. Create the certificate with the following command:
# certto o l --g enerate-certi fi cate --l o ad -pri vkey serverkey. pem --
l o ad -ca-certi fi cate cacert. pem --l o ad -ca-pri vkey cakey. pem \ -templ ate server. i nfo --o utfi l e servercert. pem
4. This results in two files being generated: serverkey.pem - The server's private key servercert.pem - The server's public key
Make sure to keep the location of the private key secret. To view the contents of the file, perform the following command:
# certto o l -i --i ni fi l e servercert. pem
When opening this file the C N= parameter should be the same as the CN that you set earlier.
For example, myco mmo nname.
5. Install the two files in the following locations:
serverkey. pem - the server's private key. Place this file in the following location:
/etc/pki /l i bvi rt/pri vate/serverkey. pem
servercert. pem - the server's certificate. Install it in the following location on the server:
/etc/pki /l i bvi rt/servercert. pem
Pro ced u re 6 .3. Issu in g a clien t cert if icat e
1. For every client (ie. any program linked with libvirt, such as virt-manager), you need to issue a certificate with the X.509 Distinguished Name (DN) set to a suitable name. This needs to be decided on a corporate level.
For example purposes the following information will be used:
C=USA,ST=North Carolina,L=Raleigh,O=Red Hat,CN=name_of_client
This process is quite similar to Procedure 6.2, “ Issuing a server certificate”
, with the following exceptions noted.
2. Make a private key with the following command:
# certto o l --g enerate-pri vkey > cl i entkey. pem
4 6
Chapt er 6 . Remot e management of guest s
3. Generate a signature for the CA's private key by first creating a template file called
cl i ent. i nfo . The file should contain the following (fields should be customized to reflect your region/location): country = USA state = North Carolina locality = Raleigh organization = Red Hat cn = client1 tls_www_client encryption_key signing_key
4. Sign the certificate with the following command:
# certto o l --g enerate-certi fi cate --l o ad -pri vkey cl i entkey. pem --
l o ad -ca-certi fi cate cacert. pem \ --l o ad -ca-pri vkey cakey. pem -templ ate cl i ent. i nfo --o utfi l e cl i entcert. pem
5. Install the certificates on the client machine:
# cp cl i entkey. pem /etc/pki /l i bvi rt/pri vate/cl i entkey. pem
# cp cl i entcert. pem /etc/pki /l i bvi rt/cl i entcert. pem
6.3. T ransport modes
For remote management, l i bvi rt supports the following transport modes:
T ran sp o rt Layer Secu rit y ( T LS)
Transport Layer Security TLS 1.0 (SSL 3.1) authenticated and encrypted TCP/IP socket, usually listening on a public port number. To use this you will need to generate client and server certificates.
The standard port is 16514.
UNIX so cket s
UNIX domain sockets are only accessible on the local machine. Sockets are not encrypted, and use
UNIX permissions or SELinux for authentication. The standard socket names are
/var/run/l i bvi rt/l i bvi rt-so ck and /var/run/l i bvi rt/l i bvi rt-so ck-ro (for read-only connections).
SSH
Transported over a Secure Shell protocol (SSH) connection. Requires Netcat (the nc package) installed. The libvirt daemon (l i bvi rtd ) must be running on the remote machine. Port 22 must be open for SSH access. You should use some sort of SSH key management (for example, the ssh-
ag ent utility) or you will be prompted for a password.
ext
The ext parameter is used for any external program which can make a connection to the remote machine by means outside the scope of libvirt. This parameter is unsupported.
T CP
4 7
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Unencrypted TCP/IP socket. Not recommended for production use, this is normally disabled, but an administrator can enable it for testing or use over a trusted network. The default port is 16509.
The default transport, if no other is specified, is TLS.
Remo t e URIs
A Uniform Resource Identifier (URI) is used by vi rsh and libvirt to connect to a remote host. URIs can also be used with the --co nnect parameter for the vi rsh command to execute single commands or migrations on remote hosts. Remote URIs are formed by taking ordinary local URIs and adding a hostname and/or transport name. As a special case, using a URI scheme of 'remote', will tell the remote libvirtd server to probe for the optimal hypervisor driver. This is equivalent to passing a NULL
URI for a local connection libvirt URIs take the general form (content in square brackets, "[]", represents optional functions): driver[+transport]://[username@ ][hostname][:port]/path[?extraparameters]
Note that if the hypervisor(driver) is QEMU, the path is mandatory. If it is XEN, it is optional.
The following are examples of valid remote URIs: qemu://hostname/ xen://hostname/ xen+ssh://hostname/
The transport method or the hostname must be provided to target an external location. For more information refer to http://libvirt.org/guide/html/Application_Development_Guide-Architecture-
Remote_URIs.html
.
Examp les o f remo t e man ag emen t p aramet ers
Connect to a remote KVM host named ho st2, using SSH transport and the SSH username
vi rtuser.The connect command for each is co nnect [<name>] [--read o nl y], where
<name> is a valid URI as explained here. For more information about the vi rsh co nnect command refer to
q emu+ ssh: //vi rtuser@ ho t2/
Connect to a remote KVM hypervisor on the host named ho st2 using TLS.
q emu: //ho st2/
T est in g examp les
Connect to the local KVM hypervisor with a non-standard UNIX socket. The full path to the UNIX socket is supplied explicitly in this case.
4 8 q emu+ uni x: ///system?so cket= /o pt/l i bvi rt/run/l i bvi rt/l i bvi rt-so ck
Connect to the libvirt daemon with an unencrypted TCP/IP connection to the server with the IP
Chapt er 6 . Remot e management of guest s
address 10.1.1.10 on port 5000. This uses the test driver with default settings.
test+ tcp: //10 . 1. 1. 10 : 50 0 0 /d efaul t
Ext ra URI p aramet ers
values must be URI-escaped (that is, a question mark (?) is appended before the parameter and special characters are converted into the URI format).
T ab le 6 .1. Ext ra URI p aramet ers
Name
name
T ran sp o rt mo d e
all modes command socket ssh and ext unix and ssh
Descrip t io n
The name passed to the remote virConnectOpen function. The name is normally formed by removing transport, hostname, port number, username and extra parameters from the remote URI, but in certain very complex cases it may be better to supply the name explicitly.
The external command.
For ext transport this is required. For ssh the default is ssh. The
PATH is searched for the command.
The path to the UNIX domain socket, which overrides the default.
For ssh transport, this is passed to the remote netcat command (see netcat).
Examp le u sag e
name=qemu:///system command=/opt/openss h/bin/ssh socket=/opt/libvirt/run/li bvirt/libvirt-sock
4 9
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Name
netcat
T ran sp o rt mo d e
ssh
Descrip t io n
The netcat command can be used to connect to remote systems. The default netcat parameter uses the nc command. For SSH transport, libvirt constructs an SSH command using the form below:
command -p port [-l
username] hostname
netcat -U socket
The port, username and hostname parameters can be specified as part of the remote URI. The
command, netcat and
socket come from other extra parameters.
Examp le u sag e
netcat=/opt/netcat/bin/n c no_verify no_tty tls ssh
If set to a non-zero value, this disables client checks of the server's certificate.
Note that to disable server checks of the client's certificate or IP address you must change the libvirtd configuration.
If set to a non-zero value, this stops ssh from asking for a password if it cannot log in to the remote machine automatically
. Use this when you do not have access to a terminal .
no_verify=1 no_tty=1
50
Chapt er 7 . O vercommit t ing wit h KVM
Chapter 7. Overcommitting with KVM
7.1. Int roduct ion
The KVM hypervisor supports overcommitting CPUs and overcommitting memory. Overcommitting is allocating more virtualized CPUs or memory than there are physical resources on the system. With
CPU overcommit, under-utilized virtualized servers or desktops can run on fewer servers which saves a number of system resources, with the net effect of less power, cooling, and investment in server hardware.
As most processes do not access 100% of their allocated memory all the time, KVM can use this behavior to its advantage and allocate more memory for guest virtual machines than the host physical machine actually has available, in a process called overcommitting of resources.
Important
Overcommitting is not an ideal solution for all memory issues as the recommended method to deal with memory shortage is to allocate less memory per guest so that the sum of all guests memory (+4G for the host O/S) is lower than the host physical machine's physical memory. If the guest virtual machines need more memory, then increase the guest virtual machines' swap space allocation. If however, should you decide to overcommit, do so with caution.
Guest virtual machines running on a KVM hypervisor do not have dedicated blocks of physical RAM assigned to them. Instead, each guest virtual machine functions as a Linux process where the host physical machine's Linux kernel allocates memory only when requested. In addition the host physical machine's memory manager can move the guest virtual machine's memory between its own physical memory and swap space. This is why overcommitting requires allotting sufficient swap space on the host physical machine to accommodate all guest virtual machines as well as enough memory for the host physical machine's processes. As a basic rule, the host physical machine's operating system requires a maximum of 4GB of memory along with a minimum of 4GB of swap
space. Refer to Example 7.1, “ Memory overcommit example” for more information.
Red Hat Knowledgebase has an article on safely and efficiently determining the size of the swap partition.
Note
The example below is provided as a guide for configuring swap only. The settings listed may not be appropriate for your environment.
Examp le 7.1. Memo ry o verco mmit examp le
This example demonstrates how to calculate swap space for overcommitting. Although it may appear to be simple in nature, the ramifications of overcommitting should not be ignored. Refer to
ExampleServer1 has 32GB of physical RAM. The system is being configured to run 50 guest virtual machines, each requiring 1GB of virtualized memory. As mentioned above, the host physical machine's system itself needs a maximum of 4GB (apart from the guest virtual machines) as well as an additional 4GB as a swap space minimum.
51
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The swap space is calculated as follows:
Calculate the amount of memory needed for the sum of all the guest virtual machines - In this example: (50 guest virtual machines * 1GB of memory per guest virtual machine) = 50GB
Add the guest virtual machine's memory amount to the amount needed for the host physical machine's OS and for the host physical machine's minimum swap space - In this example:
50GB guest virtual machine memory + 4GB host physical machine's OS + 4GB minimal swap =
58GB
Subtract this amount from the amount of physical RAM there is on the system - In this example
58GB - 32GB = 26GB
The answer is the amount of swap space that needs to be allocated. In this example 26GB
Note
Overcommitting does not work with all guest virtual machines, but has been found to work in a desktop virtualization setup with minimal intensive usage or running several identical guest virtual machines with KSM. It should be noted that configuring swap and memory overcommit is not a simple plug-in and configure formula, as each environment and setup is different.
Proceed with caution before changing these settings and make sure you completely understand your environment and setup before making any changes.
For more information on KSM and overcommitting, refer to
.
7.2. Overcommit t ing virt ualized CPUs
The KVM hypervisor supports overcommitting virtualized CPUs. Virtualized CPUs can be overcommitted as far as load limits of guest virtual machines allow. Use caution when overcommitting
VCPUs as loads near 100% may cause dropped requests or unusable response times.
Virtualized CPUs are overcommitted best when each guest virtual machine only has a single VCPU.
The Linux scheduler is very efficient with this type of load. KVM should safely support guest virtual machines with loads under 100% at a ratio of five VCPUs. Overcommitting single VCPU guest virtual machines is not an issue.
You cannot overcommit symmetric multiprocessing guest virtual machines on more than the physical number of processing cores. For example a guest virtual machine with four VCPUs should not be run on a host physical machine with a dual core processor. Overcommitting symmetric multiprocessing guest virtual machines in over the physical number of processing cores will cause significant performance degradation.
Assigning guest virtual machines VCPUs up to the number of physical cores is appropriate and works as expected. For example, running guest virtual machines with four VCPUs on a quad core host. Guest virtual machines with less than 100% loads should function effectively in this setup.
52
Chapt er 7 . O vercommit t ing wit h KVM
Important
Do not overcommit memory or CPUs in a production environment without extensive testing.
Applications which use 100% of memory or processing resources may become unstable in overcommitted environments. Test before deploying.
53
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 8. KSM
The concept of shared memory is common in modern operating systems. For example, when a program is first started it shares all of its memory with the parent program. When either the child or parent program tries to modify this memory, the kernel allocates a new memory region, copies the original contents and allows the program to modify this new region. This is known as copy on write.
KSM is a new Linux feature which uses this concept in reverse. KSM enables the kernel to examine two or more already running programs and compare their memory. If any memory regions or pages are identical, KSM reduces multiple identical memory pages to a single page. This page is then marked copy on write. If the contents of the page is modified by a guest virtual machine, a new page is created for that guest virtual machine.
This is useful for virtualization with KVM. When a guest virtual machine is started, it inherits only the memory from the parent q emu-kvm process. Once the guest virtual machine is running the contents of the guest virtual machine operating system image can be shared when guests are running the same operating system or applications. KSM only identifies and merges identical pages which does not interfere with the guest virtual machine or impact the security of the host physical machine or the guests. KSM allows KVM to request that these identical guest virtual machine memory regions be shared.
KSM provides enhanced memory speed and utilization. With KSM, common process data is stored in cache or in main memory. This reduces cache misses for the KVM guests which can improve performance for some applications and operating systems. Secondly, sharing memory reduces the overall memory usage of guests which allows for higher densities and greater utilization of resources.
Note
Starting in Red Hat Enterprise Linux 6.5, KSM is NUMA aware. This allows it to take NUMA locality into account while coalescing pages, thus preventing performance drops related to pages being moved to a remote node. Red Hat recommends avoiding cross-node memory merging when KSM is in use. If KSM is in use, change the
/sys/kernel /mm/ksm/merg e_acro ss_no d es tunable to 0 to avoid merging pages across
NUMA nodes. Kernel memory accounting statistics can eventually contradict each other after large amounts of cross-node merging. As such, numad can become confused after the KSM daemon merges large amounts of memory. If your system has a large amount of free memory, you may achieve higher performance by turning off and disabling the KSM daemon. Refer to the Red Hat Enterprise Linux Performance Tuning Guide for more information on NUMA.
Red Hat Enterprise Linux uses two separate methods for controlling KSM:
The ksm service starts and stops the KSM kernel thread.
The ksmtuned service controls and tunes the ksm, dynamically managing same-page merging.
The ksmtuned service starts ksm and stops the ksm service if memory sharing is not necessary.
The ksmtuned service must be told with the retune parameter to run when new guests are created or destroyed.
Both of these services are controlled with the standard service management tools.
T h e KSM service
54
Chapt er 8 . KSM
The ksm service is included in the qemu-kvm package. KSM is off by default on Red Hat Enterprise
Linux 6. When using Red Hat Enterprise Linux 6 as a KVM host physical machine, however, it is likely turned on by the ksm/ksmtuned services.
When the ksm service is not started, KSM shares only 2000 pages. This default is low and provides limited memory saving benefits.
When the ksm service is started, KSM will share up to half of the host physical machine system's main memory. Start the ksm service to enable KSM to share more memory.
# service ksm start
Starting ksm: [ OK ]
The ksm service can be added to the default startup sequence. Make the ksm service persistent with the chkconfig command.
# chkconfig ksm on
T h e KSM t u n in g service
The ksmtuned service does not have any options. The ksmtuned service loops and adjusts ksm.
The ksmtuned service is notified by libvirt when a guest virtual machine is created or destroyed.
# service ksmtuned start
Starting ksmtuned: [ OK ]
The ksmtuned service can be tuned with the retune parameter. The retune parameter instructs
ksmtuned to run tuning functions manually.
Before changing the parameters in the file, there are a few terms that need to be clarified:
thres - Activation threshold, in kbytes. A KSM cycle is triggered when the thres value added to the sum of all qemu-kvm processes RSZ exceeds total system memory. This parameter is the equivalent in kbytes of the percentage defined in KSM_THRES_COEF.
The /etc/ksmtuned . co nf file is the configuration file for the ksmtuned service. The file output below is the default ksmtuned . co nf file.
# Configuration file for ksmtuned.
# How long ksmtuned should sleep between tuning adjustments
# KSM_MONITOR_INTERVAL=60
# Millisecond sleep between ksm scans for 16Gb server.
# Smaller servers sleep more, bigger sleep less.
# KSM_SLEEP_MSEC=10
# KSM_NPAGES_BOOST is added to the npages value, when free memory is less than thres.
# KSM_NPAGES_BOOST=300
# KSM_NPAGES_DECAY Value given is subtracted to the npages value, when
free memory is greater than thres.
# KSM_NPAGES_DECAY=-50
# KSM_NPAGES_MIN is the lower limit for the npages value.
55
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# KSM_NPAGES_MIN=64
# KSM_NAGES_MAX is the upper limit for the npages value.
# KSM_NPAGES_MAX=1250
# KSM_TRES_COEF - is the RAM percentage to be calculated in parameter
thres.
# KSM_THRES_COEF=20
# KSM_THRES_CONST - If this is a low memory system, and the thres value is less than KSM_THRES_CONST, then reset thres value to KSM_THRES_CONST value.
# KSM_THRES_CONST=2048
# uncomment the following to enable ksmtuned debug information
# LOGFILE=/var/log/ksmtuned
# DEBUG=1
KSM variab les an d mo n it o rin g
KSM stores monitoring data in the /sys/kernel /mm/ksm/ directory. Files in this directory are updated by the kernel and are an accurate record of KSM usage and statistics.
The variables in the list below are also configurable variables in the /etc/ksmtuned . co nf file as noted below.
T h e /sys/kernel /mm/ksm/ f iles f u ll_scan s
Full scans run.
p ag es_sh ared
Total pages shared.
p ag es_sh arin g
Pages presently shared.
p ag es_t o _scan
Pages not scanned.
p ag es_u n sh ared
Pages no longer shared.
p ag es_vo lat ile
Number of volatile pages.
ru n
Whether the KSM process is running.
sleep _millisecs
Sleep milliseconds.
56
Chapt er 8 . KSM
KSM tuning activity is stored in the /var/l o g /ksmtuned log file if the DEBUG=1 line is added to the
/etc/ksmtuned . co nf file. The log file location can be changed with the LOGFILE parameter.
Changing the log file location is not advised and may require special configuration of SELinux settings.
Deact ivat in g KSM
KSM has a performance overhead which may be too large for certain environments or host physical machine systems.
KSM can be deactivated by stopping the ksmtuned and the ksm service. Stopping the services deactivates KSM but does not persist after restarting.
# service ksmtuned stop
Stopping ksmtuned: [ OK ]
# service ksm stop
Stopping ksm: [ OK ]
Persistently deactivate KSM with the chkco nfi g command. To turn off the services, run the following commands:
# chkconfig ksm off
# chkconfig ksmtuned off
Important
Ensure the swap size is sufficient for the committed RAM even with KSM. KSM reduces the RAM usage of identical or similar guests. Overcommitting guests with KSM without sufficient swap space may be possible but is not recommended because guest virtual machine memory use can result in pages becoming unshared.
57
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 9. Advanced guest virtual machine administration
This chapter covers advanced administration tools for fine tuning and controlling system resources as they are made available to guest virtual machines.
9.1. Cont rol Groups (cgroups)
Red Hat Enterprise Linux 6 provides a new kernel feature: control groups, which are often referred to as
cgroups. Cgroups allow you to allocate resources such as CPU time, system memory, network bandwidth, or a combination of these resources among user-defined groups of tasks (processes) running on a system. You can monitor the cgroups you configure, deny cgroups access to certain resources, and even reconfigure your cgroups dynamically on a running system.
The cgroup functionality is fully supported by lib virt . By default, lib virt puts each guest into a separate control group for various controllers (such as memory, cpu, blkio, device).
When a guest is started, it is already in a cgroup. The only configuration that may be required is the setting of policies on the cgroups. Refer to the Red Hat Enterprise Linux Resource Management Guide for more information on cgroups.
9.2. Hugepage support
In t ro d u ct io n
x86 CPUs usually address memory in 4kB pages, but they are capable of using larger pages known as hugepages. KVM guests can be deployed with hug epag e memory support in order to improve performance by increasing CPU cache hits against the Transaction Lookaside Buffer (TLB).
hug epag es can significantly increase performance, particularly for large memory and memoryintensive workloads. Red Hat Enterprise Linux 6 is able to more effectively manage large amounts of memory by increasing the page size through the use of hugepages.
By using hugepages for a KVM guest, less memory is used for page tables and TLB (Translation
Lookaside Buffer) misses are reduced, thereby significantly increasing performance, especially for memory-intensive situations.
T ran sp aren t Hu g ep ag e Su p p o rt is a kernel feature that reduces TLB entries needed for an application. By also allowing all free memory to be used as cache, performance is increased.
Usin g T ran sp aren t Hu g ep ag e Su p p o rt
To use Transparent Hugepage Support, no special configuration in the q emu. co nf file is required.
Hugepages are used by default if /sys/kernel /mm/red hat_transparent_hug epag e/enabl ed is set to always.
Transparent Hugepage Support does not prevent the use of hugetlbfs. However, when hugetlbfs is not used, KVM will use transparent hugepages instead of the regular 4kB page size.
9.3. Running Red Hat Ent erprise Linux as a guest virt ual machine on a
Hyper-V hypervisor
58
Chapt er 9 . Advanced guest virt ual machine administ rat ion
It is possible to run a Red Hat Enterprise Linux guest virtual machine on a Microsoft Windows host physical machine running the Microsoft Windows Hyper-V hypervisor. In particular, the following enhancements have been made to allow for easier deployment and management of Red Hat
Enterprise Linux guest virtual machines:
Upgraded VMBUS protocols - VMBUS protocols have been upgraded to Windows 8 level. As part of this work, now VMBUS interrupts can be processed on all available virtual CPUs in the guest.
Furthermore, the signaling protocol between the Red Hat Enterprise Linux guest virtual machine and the Windows host physical machine has been optimized.
Synthetic frame buffer driver - Provides enhanced graphics performance and superior resolution for Red Hat Enterprise Linux desktop users.
Live Virtual Machine Backup support - Provisions uninterrupted backup support for live Red Hat
Enterprise Linux guest virtual machines.
Dynamic expansion of fixed size Linux VHDs - Allows expansion of live mounted fixed size Red
Hat Enterprise Linux VHDs.
For more information, refer to the following article:" Enabling Linux Support on Windows Server 2012
R2 Hyper-V ".
9.4 . Guest virt ual machine memory allocat ion
The following procedure shows how to allocate memory for a guest virtual machine. This allocation and assignment works only at boot time and any changes to any of the memory values will not take effect until the next reboot. The maximum memory that can be allocated per guest is 4 TiB, providing that this memory allocation isn't more than what the host physical machine resources can provide.
Valid memory units include:
b or bytes for bytes
KB for kilobytes (10
3
or blocks of 1,000 bytes)
k or KiB for kibibytes (2
10
or blocks of 1024 bytes)
MB for megabytes (10
6
or blocks of 1,000,000 bytes)
M or MiB for mebibytes (2
20
or blocks of 1,048,576 bytes)
GB for gigabytes (10
9
or blocks of 1,000,000,000 bytes)
G or GiB for gibibytes (2
30
or blocks of 1,073,741,824 bytes)
TB for terabytes (10
12
or blocks of 1,000,000,000,000 bytes)
T or TiB for tebibytes (2
40
or blocks of 1,099,511,627,776 bytes)
Note that all values will be rounded up to the nearest kibibyte by libvirt, and may be further rounded to the granularity supported by the hypervisor. Some hypervisors also enforce a minimum, such as
4000KiB (or 4000 x 2
10
or 4,096,000 bytes). The units for this value are determined by the optional attribute memory unit, which defaults to the kibibytes (KiB) as a unit of measure where the value given is multiplied by 2
10
or blocks of 1024 bytes.
59
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
In the cases where the guest virtual machine crashes the optional attribute dumpCore can be used to control whether the guest virtual machine's memory should be included in the generated coredump
(dumpCore='on') or not included (dumpCore='off'). Note that the default setting is on so if the parameter is not set to off, the guest virtual machine memory will be included in the coredump file.
The currentMemory attribute determines the actual memory allocation for a guest virtual machine.
This value can be less than the maximum allocation, to allow for ballooning up the guest virtual machines memory on the fly. If this is omitted, it defaults to the same value as the memory element.
The unit attribute behaves the same as for memory.
In all cases for this section, the domain XML needs to be altered as follows:
<domain>
<memory unit='KiB' dumpCore='off'>524288</memory>
<!-- changes the memory unit to KiB and does not allow the guest virtual machine's memory to be included in the generated coredump file -
->
<currentMemory unit='KiB'>524288</currentMemory>
<!-- makes the current memory unit 524288 KiB -->
...
</domain>
9.5. Aut omat ically st art ing guest virt ual machines
This section covers how to make guest virtual machines start automatically during the host physical machine system's boot phase.
This example uses vi rsh to set a guest virtual machine, TestServer, to automatically start when the host physical machine boots.
# virsh autostart TestServer
Domain TestServer marked as autostarted
The guest virtual machine now automatically starts with the host physical machine.
To stop a guest virtual machine automatically booting use the --disable parameter
# virsh autostart --disable TestServer
Domain TestServer unmarked as autostarted
The guest virtual machine no longer automatically starts with the host physical machine.
9.6. Disable SMART disk monit oring for guest virt ual machines
SMART disk monitoring can be safely disabled as virtual disks and the physical storage devices are managed by the host physical machine.
# service smartd stop
# chkconfig --del smartd
9.7. Configuring a VNC Server
60
Chapt er 9 . Advanced guest virt ual machine administ rat ion
To configure a VNC server, use the Remo t e Deskt o p application in Syst em > Pref eren ces.
Alternatively, you can run the vi no -preferences command.
Use the following step set up a dedicated VNC server session:
If needed, Create and then Edit the ~ /. vnc/xstartup file to start a GNOME session whenever
vn cserver is started. The first time you run the vn cserver script it will ask you for a password you want to use for your VNC session. For more information on vnc server files refer to the Red Hat
Enterprise Linux Installation Guide.
9.8. Generat ing a new unique MAC address
In some cases you will need to generate a new and unique MAC address for a guest virtual machine.
There is no command line tool available to generate a new MAC address at the time of writing. The script provided below can generate a new MAC address for your guest virtual machines. Save the script on your guest virtual machine as macg en. py. Now from that directory you can run the script using . /macg en. py and it will generate a new MAC address. A sample output would look like the following:
$ ./macgen.py
00:16:3e:20:b0:11
#!/usr/bin/python
# macgen.py script to generate a MAC address for guest virtual machines
# import random
# def randomMAC():
mac = [ 0x00, 0x16, 0x3e,
random.randint(0x00, 0x7f),
random.randint(0x00, 0xff),
random.randint(0x00, 0xff) ]
return ':'.join(map(lambda x: "%02x" % x, mac))
# print randomMAC()
9.8.1. Anot her met hod t o generat e a new MAC for your guest virt ual machine
You can also use the built-in modules of pytho n-vi rti nst to generate a new MAC address and
UUID for use in a guest virtual machine configuration file:
# echo 'import virtinst.util ; print\
virtinst.util.uuidToString(virtinst.util.randomUUID())' | python
# echo 'import virtinst.util ; print virtinst.util.randomMAC()' | python
The script above can also be implemented as a script file as seen below.
#!/usr/bin/env python
# -*- mode: python; -*print "" print "New UUID:" import virtinst.util ; print
61
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
virtinst.util.uuidToString(virtinst.util.randomUUID()) print "New MAC:" import virtinst.util ; print virtinst.util.randomMAC() print ""
9.9. Improving guest virt ual machine response t ime
Guest virtual machines can sometimes be slow to respond with certain workloads and usage patterns. Examples of situations which may cause slow or unresponsive guest virtual machines:
Severely overcommitted memory.
Overcommitted memory with high processor usage
Other (not q emu-kvm processes) busy or stalled processes on the host physical machine.
KVM guest virtual machines function as Linux processes. Linux processes are not permanently kept in main memory (physical RAM) and will be placed into swap space (virtual memory) especially if they are not being used. If a guest virtual machine is inactive for long periods of time, the host physical machine kernel may move the guest virtual machine into swap. As swap is slower than physical memory it may appear that the guest is not responding. This changes once the guest is loaded into the main memory. Note that the process of loading a guest virtual machine from swap to main memory may take several seconds per gigabyte of RAM assigned to the guest virtual machine, depending on the type of storage used for swap and the performance of the components.
KVM guest virtual machines processes may be moved to swap regardless of whether memory is overcommitted or overall memory usage.
Using unsafe overcommit levels or overcommitting with swap turned off guest virtual machine processes or other critical processes is not recommended. Always ensure the host physical machine has sufficient swap space when overcommitting memory.
For more information on overcommitting with KVM, refer to Section 7.1, “ Introduction” .
Warning
Virtual memory allows a Linux system to use more memory than there is physical RAM on the system. Underused processes are swapped out which allows active processes to use memory, improving memory utilization. Disabling swap reduces memory utilization as all processes are stored in physical RAM.
If swap is turned off, do not overcommit guest virtual machines. Overcommitting guest virtual machines without any swap can cause guest virtual machines or the host physical machine system to crash.
9.10. Virt ual machine t imer management wit h libvirt
Accurate time keeping on guest virtual machines is a key challenge for virtualization platforms.
Different hypervisors attempt to handle the problem of time keeping in a variety of ways. libvirt provides hypervisor independent configuration settings for time management, using the <clock> and
<timer> elements in the domain XML. The domain XML can be edited using the vi rsh ed i t
command. See Section 15.6, “ Editing a guest virtual machine's configuration file” for details.
62
Chapt er 9 . Advanced guest virt ual machine administ rat ion
The <cl o ck> element is used to determine how the guest virtual machine clock is synchronized with the host physical machine clock. The clock element has the following attributes:
o ffset determines how the guest virtual machine clock is offset from the host physical machine clock. The offset attribute has the following possible values:
T ab le 9 .1. O f f set at t rib u t e valu es
Valu e
utc localtime timezone variable
Descrip t io n
The guest virtual machine clock will be synchronized to UTC when booted.
The guest virtual machine clock will be synchronized to the host physical machine's configured timezone when booted, if any.
The guest virtual machine clock will be synchronized to a given timezone, specified by the timezone attribute.
The guest virtual machine clock will be synchronized to an arbitrary offset from UTC.
The delta relative to UTC is specified in seconds, using the adjustment attribute. The guest virtual machine is free to adjust the Real
Time Clock (RTC) over time and expect that it will be honored following the next reboot. This is in contrast to utc mode, where any RTC adjustments are lost at each reboot.
Note
The value u t c is set as the clock offset in a virtual machine by default. However, if the guest virtual machine clock is run with the lo calt ime value, the clock offset needs to be changed to a different value in order to have the guest virtual machine clock synchronized with the host physical machine clock.
The ti mezo ne attribute determines which timezone is used for the guest virtual machine clock.
The ad justment attribute provides the delta for guest virtual machine clock synchronization. In seconds, relative to UTC.
Examp le 9 .1. Always syn ch ro n iz e t o UT C
<clock offset="utc" />
Examp le 9 .2. Always syn ch ro n iz e t o t h e h o st p h ysical mach in e t imez o n e
<clock offset="localtime" />
Examp le 9 .3. Syn ch ro n iz e t o an arb it rary t imez o n e
63
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<clock offset="timezone" timezone="Europe/Paris" />
Examp le 9 .4 . Syn ch ro n iz e t o UT C + arb it rary o f f set
<clock offset="variable" adjustment="123456" />
9.10.1. t imer child element for clock
A clock element can have zero or more timer elements as children. The timer element specifies a time source used for guest virtual machine clock synchronization. The timer element has the following attributes. Only the name is required, all other attributes are optional.
The name attribute dictates the type of the time source to use, and can be one of the following:
T ab le 9 .2. n ame at t rib u t e valu es
Valu e
pit rtc tsc kvmclock
Descrip t io n
Programmable Interval Timer - a timer with periodic interrupts.
Real Time Clock - a continuously running timer with periodic interrupts.
Time Stamp Counter - counts the number of ticks since reset, no interrupts.
KVM clock - recommended clock source for KVM guest virtual machines. KVM pvclock, or kvmclock lets guest virtual machines read the host physical machine’s wall clock time.
9.10.2. t rack
The track attribute specifies what is tracked by the timer. Only valid for a name value of rtc.
T ab le 9 .3. t rack at t rib u t e valu es
Valu e
boot guest wall
Descrip t io n
Corresponds to old host physical machine option, this is an unsupported tracking option.
RTC always tracks guest virtual machine time.
RTC always tracks host time.
9.10.3. t ickpolicy
The ti ckpo l i cy attribute assigns the policy used to pass ticks on to the guest virtual machine. The following values are accepted:
T ab le 9 .4 . t ickp o licy at t rib u t e valu es
Valu e Descrip t io n
64
Chapt er 9 . Advanced guest virt ual machine administ rat ion
Valu e
delay catchup merge discard
Descrip t io n
Continue to deliver at normal rate (i.e. ticks are delayed).
Deliver at a higher rate to catch up.
Ticks merged into one single tick.
All missed ticks are discarded.
9.10.4 . frequency, mode, and present
The freq uency attribute is used to set a fixed frequency, and is measured in Hz. This attribute is only relevant when the name element has a value of tsc. All other timers operate at a fixed frequency
(pi t, rtc).
mo d e determines how the time source is exposed to the guest virtual machine. This attribute is only relevant for a name value of tsc. All other timers are always emulated. Command is as follows:
<ti mer name= ' tsc' freq uency= ' NNN' mo d e= ' auto | nati ve| emul ate| smpsafe' />.
Mode definitions are given in the table.
T ab le 9 .5. mo d e at t rib u t e valu es
Valu e
auto native emulate smpsafe
Descrip t io n
Native if TSC is unstable, otherwise allow native
TSC access.
Always allow native TSC access.
Always emulate TSC.
Always emulate TSC and interlock SMP
present is used to override the default set of timers visible to the guest virtual machine..
T ab le 9 .6 . p resen t at t rib u t e valu es
Valu e
yes no
Descrip t io n
Force this timer to the visible to the guest virtual machine.
Force this timer to not be visible to the guest virtual machine.
9.10.5. Examples using clock synchroniz at ion
Examp le 9 .5. Clo ck syn ch ro n iz in g t o lo cal t ime wit h RT C an d PIT t imers
In this example. the clock is synchronized to local time with RTC and PIT timers
<clock offset="localtime">
<timer name="rtc" tickpolicy="catchup" track="guest virtual machine"
/>
<timer name="pit" tickpolicy="delay" />
</clock>
65
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
The PIT clocksource can be used with a 32-bit guest running under a 64-bit host (which cannot use PIT), with the following conditions:
Guest virtual machine may have only one CPU
APIC timer must be disabled (use the "noapictimer" command line option)
NoHZ mode must be disabled in the guest (use the "nohz=off" command line option)
High resolution timer mode must be disabled in the guest (use the "highres=off" command line option)
The PIT clocksource is not compatible with either high resolution timer mode, or NoHz mode.
9.11. Using PMU t o monit or guest virt ual machine performance
In Red Hat Enterprise Linux 6.4, vPMU (virtual PMU )was introduced as technical-preview. vPMU is based on Intel's PMU (Performance Monitoring Units) and may only be used on Intel machines. PMU allows the tracking of statistics which indicate how a guest virtual machine is functioning.
Using performance monitoring, allows developers to use the CPU's PMU counter while using the performance tool for profiling. The virtual performance monitoring unit feature allows virtual machine users to identify sources of possible performance problems in their guest virtual machines, thereby improving the ability to profile a KVM guest virtual machine.
To enable the feature, the -cpu ho st flag must be set.
This feature is only supported with guest virtual machines running Red Hat Enterprise Linux 6 and is disabled by default. This feature only works using the Linux perf tool. Make sure the perf package is installed using the command:
# yum i nstal l perf.
See the man page on perf for more information on the perf commands.
9.12. Guest virt ual machine power management
It is possible to forcibly enable or disable BIOS advertisements to the guest virtual machine's operating system by changing the following parameters in the Domain XML for Libvirt:
...
<pm>
<suspend-to-disk enabled='no'/>
<suspend-to-mem enabled='yes'/>
</pm>
...
The element pm enables ('yes') or disables ('no') BIOS support for S3 (suspend-to-disk) and S4
(suspend-to-mem) ACPI sleep states. If nothing is specified, then the hypervisor will be left with its default value.
66
Chapt er 1 0 . G uest virt ual machine device configurat ion
Chapter 10. Guest virtual machine device configuration
Red Hat Enterprise Linux 6 supports three classes of devices for guest virtual machines:
Emulated devices are purely virtual devices that mimic real hardware, allowing unmodified guest operating systems to work with them using their standard in-box drivers. Red Hat Enterprise Linux
6 supports up to 216 virtio devices.
Virtio devices are purely virtual devices designed to work optimally in a virtual machine. Virtio devices are similar to emulated devices, however, non-Linux virtual machines do not include the drivers they require by default. Virtualization management software like the Virtual Machine
Manager (virt - man ag er) and the Red Hat Enterprise Virtualization Hypervisor install these drivers automatically for supported non-Linux guest operating systems. Red Hat Enterprise Linux
6 supports up to 700 scsi disks.
Assigned devices are physical devices that are exposed to the virtual machine. This method is also known as 'passthrough'. Device assignment allows virtual machines exclusive access to PCI devices for a range of tasks, and allows PCI devices to appear and behave as if they were physically attached to the guest operating system. Red Hat Enterprise Linux 6 supports up to 32 assigned devices per virtual machine.
Device assignment is supported on PCIe devices, including select graphics devices. Nvidia Kseries Quadro, GRID, and Tesla graphics card GPU functions are now supported with device assignment in Red Hat Enterprise Linux 6. Parallel PCI devices may be supported as assigned devices, but they have severe limitations due to security and system configuration conflicts.
Red Hat Enterprise Linux 6 supports PCI hotplug of devices exposed as single function slots to the virtual machine. Single function host devices and individual functions of multi-function host devices may be configured to enable this. Configurations exposing devices as multi-function PCI slots to the virtual machine are recommended only for non-hotplug applications.
Note
Platform support for interrupt remapping is required to fully isolate a guest with assigned devices from the host. Without such support, the host may be vulnerable to interrupt injection attacks from a malicious guest. In an environment where guests are trusted, the admin may opt-in to still allow PCI device assignment using the al l o w_unsafe_i nterrupts option to the vfio_iommu_type1 module. This may either be done persistently by adding a .conf file
(e.g. l o cal . co nf) to /etc/mo d pro be. d containing the following: options vfio_iommu_type1 allow_unsafe_interrupts=1 or dynamically using the sysfs entry to do the same:
# echo 1 >
/sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts
10.1. PCI devices
67
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
PCI device assignment is only available on hardware platforms supporting either Intel VT-d or AMD
IOMMU. These Intel VT-d or AMD IOMMU specifications must be enabled in BIOS for PCI device assignment to function.
Pro ced u re 10.1. Prep arin g an In t el syst em f o r PCI d evice assig n men t
1. En ab le t h e In t el VT - d sp ecif icat io n s
The Intel VT-d specifications provide hardware support for directly assigning a physical device to a virtual machine. These specifications are required to use PCI device assignment with Red Hat Enterprise Linux.
The Intel VT-d specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default. The terms used to refer to these specifications can differ between manufacturers; consult your system manufacturer's documentation for the appropriate terms.
2. Act ivat e In t el VT - d in t h e kern el
Activate Intel VT-d in the kernel by adding the intel_iommu=on parameter to the end of the
GRUB_CMDLINX_LINUX line, within the quotes, in the /etc/sysco nfi g /g rub file.
The example below is a modified g rub file with Intel VT-d activated.
GRUB_CMDLINE_LINUX="rd.lvm.lv=vg_VolGroup00/LogVol01 vconsole.font=latarcyrheb-sun16 rd.lvm.lv=vg_VolGroup_1/root vconsole.keymap=us $([ -x /usr/sbin/rhcrashkernel-param ] & &
/usr/sbin/ rhcrashkernel-param || :) rhgb quiet i ntel _i o mmu= o n"
3. Reg en erat e co n f ig f ile
Regenerate /boot/grub2/grub.cfg by running: grub2-mkconfig -o /boot/grub2/grub.cfg
4. Read y t o u se
Reboot the system to enable the changes. Your system is now capable of PCI device assignment.
Pro ced u re 10.2. Prep arin g an AMD syst em f o r PCI d evice assig n men t
1. En ab le t h e AMD IO MMU sp ecif icat io n s
The AMD IOMMU specifications are required to use PCI device assignment in Red Hat
Enterprise Linux. These specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default.
2. En ab le IO MMU kern el su p p o rt
Append amd_iommu=on to the end of the GRUB_CMDLINX_LINUX line, within the quotes, in
/etc/sysco nfi g /g rub so that AMD IOMMU specifications are enabled at boot.
3. Reg en erat e co n f ig f ile
68
Chapt er 1 0 . G uest virt ual machine device configurat ion
Regenerate /boot/grub2/grub.cfg by running: grub2-mkconfig -o /boot/grub2/grub.cfg
4. Read y t o u se
Reboot the system to enable the changes. Your system is now capable of PCI device assignment.
10.1.1. Assigning a PCI device wit h virsh
These steps cover assigning a PCI device to a virtual machine on a KVM hypervisor.
This example uses a PCIe network controller with the PCI identifier code, pci _0 0 0 0 _0 1_0 0 _0 , and a fully virtualized guest machine named guest1-rhel7-64.
Pro ced u re 10.3. Assig n in g a PCI d evice t o a g u est virt u al mach in e wit h virsh
1. Id en t if y t h e d evice
First, identify the PCI device designated for device assignment to the virtual machine. Use the
l spci command to list the available PCI devices. You can refine the output of l spci with
g rep.
This example uses the Ethernet controller highlighted in the following output:
# lspci | grep Ethernet
0 0 : 19 . 0 Ethernet co ntro l l er: Intel C o rpo rati o n 8256 7LM-2 G i g abi t
Netwo rk C o nnecti o n
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
This Ethernet controller is shown with the short identifier 0 0 : 19 . 0 . We need to find out the full identifier used by vi rsh in order to assign this PCI device to a virtual machine.
To do so, use the vi rsh no d ed ev-l i st command to list all devices of a particular type
(pci ) that are attached to the host machine. Then look at the output for the string that maps to the short identifier of the device you wish to use.
This example highlights the string that maps to the Ethernet controller with the short identifier
0 0 : 19 . 0 . In this example, the : and . characters are replaced with underscores in the full identifier.
# virsh nodedev-list --cap pci pci_0000_00_00_0 pci_0000_00_01_0 pci_0000_00_03_0 pci_0000_00_07_0 pci_0000_00_10_0 pci_0000_00_10_1 pci_0000_00_14_0 pci_0000_00_14_1 pci_0000_00_14_2
69
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
pci_0000_00_14_3 pci_0000_0 0 _19 _0 pci_0000_00_1a_0 pci_0000_00_1a_1 pci_0000_00_1a_2 pci_0000_00_1a_7 pci_0000_00_1b_0 pci_0000_00_1c_0 pci_0000_00_1c_1 pci_0000_00_1c_4 pci_0000_00_1d_0 pci_0000_00_1d_1 pci_0000_00_1d_2 pci_0000_00_1d_7 pci_0000_00_1e_0 pci_0000_00_1f_0 pci_0000_00_1f_2 pci_0000_00_1f_3 pci_0000_01_00_0 pci_0000_01_00_1 pci_0000_02_00_0 pci_0000_02_00_1 pci_0000_06_00_0 pci_0000_07_02_0 pci_0000_07_03_0
Record the PCI device number that maps to the device you want to use; this is required in other steps.
2. Review d evice in f o rmat io n
Information on the domain, bus, and function are available from output of the vi rsh
no d ed ev-d umpxml command: virsh nodedev-dumpxml pci_0000_00_19_0
<device>
<name>pci_0000_00_19_0</name>
<parent>computer</parent>
<driver>
<name>e1000e</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>0</bus>
<slot>25</slot>
<function>0</function>
<product id='0x1502'>82579LM Gigabit Network
Connection</product>
<vendor id='0x8086'>Intel Corporation</vendor>
<iommuGroup number='7'>
<address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
</iommuGroup>
</capability>
</device>
70
Chapt er 1 0 . G uest virt ual machine device configurat ion
Note
An IOMMU group is determined based on the visibility and isolation of devices from the perspective of the IOMMU. Each IOMMU group may contain one or more devices. When multiple devices are present, all endpoints within the IOMMU group must be claimed for any device within the group to be assigned to a guest. This can be accomplished either by also assigning the extra endpoints to the guest or by detaching them from the host driver using vi rsh no d ed ev-d etach. Devices contained within a single group may not be split between multiple guests or split between host and guest. Nonendpoint devices such as PCIe root ports, switch ports, and bridges should not be detached from the host drivers and will not interfere with assignment of endpoints.
Devices within an IOMMU group can be determined using the iommuGroup section of the vi rsh no d ed ev-d umpxml output. Each member of the group is provided via a separate "address" field. This information may also be found in sysfs using the following:
$ ls /sys/bus/pci/devices/0000:01:00.0/iommu_group/devices/
An example of the output from this would be:
0000:01:00.0 0000:01:00.1
To assign only 0000.01.00.0 to the guest, the unused endpoint should be detached from the host before starting the guest:
$ virsh nodedev-detach pci_0000_01_00_1
3. Det ermin e req u ired co n f ig u rat io n d et ails
Refer to the output from the vi rsh no d ed ev-d umpxml pci _0 0 0 0 _0 0 _19 _0 command for the values required for the configuration file.
The example device has the following values: bus = 0, slot = 25 and function = 0. The decimal configuration uses those three values: bus='0' slot='25' function='0'
4. Ad d co n f ig u rat io n d et ails
Run vi rsh ed i t, specifying the virtual machine name, and add a device entry in the
<so urce> section to assign the PCI device to the guest virtual machine.
# virsh edit guest1-rhel7-64
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0' bus='0' slot='25' function='0'/>
</source>
</hostdev>
71
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Alternately, run vi rsh attach-d evi ce, specifying the virtual machine name and the guest's XML file: virsh attach-device guest1-rhel6-64 fi l e. xml
5. St art t h e virt u al mach in e
# virsh start guest1-rhel6-64
The PCI device should now be successfully assigned to the virtual machine, and accessible to the guest operating system.
10.1.2. Assigning a PCI device wit h virt -manager
PCI devices can be added to guest virtual machines using the graphical vi rt-manag er tool. The following procedure adds a Gigabit Ethernet controller to a guest virtual machine.
Pro ced u re 10.4 . Assig n in g a PCI d evice t o a g u est virt u al mach in e u sin g virt - man ag er
1. O p en t h e h ard ware set t in g s
Open the guest virtual machine and click the Ad d Hard ware button to add a new device to the virtual machine.
72
Fig u re 10.1. T h e virt u al mach in e h ard ware in f o rmat io n win d o w
Chapt er 1 0 . G uest virt ual machine device configurat ion
2. Select a PCI d evice
Select PCI Ho st Device from the Hard ware list on the left.
Select an unused PCI device. If you select a PCI device that is in use by another guest an error may result. In this example, a spare 82576 network device is used. Click Fi ni sh to complete setup.
Fig u re 10.2. T h e Ad d n ew virt u al h ard ware wiz ard
3. Ad d t h e n ew d evice
The setup is complete and the guest virtual machine now has direct access to the PCI device.
73
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 10.3. T h e virt u al mach in e h ard ware in f o rmat io n win d o w
Note
If device assignment fails, there may be other endpoints in the same IOMMU group that are still attached to the host. There is no way to retrieve group information using virt-manager, but virsh commands can be used to analyze the bounds of the IOMMU group and if necessary sequester devices.
Section 10.1.1, “ Assigning a PCI device with virsh”
for more information on
IOMMU groups and how to detach endpoint devices using virsh.
10.1.3. PCI device assignment wit h virt -inst all
To use virt - in st all to assign a PCI device, use the --host-device parameter.
Pro ced u re 10.5. Assig n in g a PCI d evice t o a virt u al mach in e wit h virt - in st all
1. Id en t if y t h e d evice
74
Chapt er 1 0 . G uest virt ual machine device configurat ion
Identify the PCI device designated for device assignment to the guest virtual machine.
# lspci | grep Ethernet
00:19.0 Ethernet controller: Intel Corporation 82567LM-2 Gigabit
Network Connection
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
The vi rsh no d ed ev-l i st command lists all devices attached to the system, and identifies each PCI device with a string. To limit output to only PCI devices, run the following command:
# virsh nodedev-list --cap pci pci_0000_00_00_0 pci_0000_00_01_0 pci_0000_00_03_0 pci_0000_00_07_0 pci_0000_00_10_0 pci_0000_00_10_1 pci_0000_00_14_0 pci_0000_00_14_1 pci_0000_00_14_2 pci_0000_00_14_3 pci_0000_00_19_0 pci_0000_00_1a_0 pci_0000_00_1a_1 pci_0000_00_1a_2 pci_0000_00_1a_7 pci_0000_00_1b_0 pci_0000_00_1c_0 pci_0000_00_1c_1 pci_0000_00_1c_4 pci_0000_00_1d_0 pci_0000_00_1d_1 pci_0000_00_1d_2 pci_0000_00_1d_7 pci_0000_00_1e_0 pci_0000_00_1f_0 pci_0000_00_1f_2 pci_0000_00_1f_3 pci_0000_01_00_0 pci_0000_01_00_1 pci_0000_02_00_0 pci_0000_02_00_1 pci_0000_06_00_0 pci_0000_07_02_0 pci_0000_07_03_0
Record the PCI device number; the number is needed in other steps.
Information on the domain, bus and function are available from output of the vi rsh
no d ed ev-d umpxml command:
# virsh nodedev-dumpxml pci_0000_01_00_0
75
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<device>
<name>pci_0000_01_00_0</name>
<parent>pci_0000_00_01_0</parent>
<driver>
<name>igb</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>1</bus>
<slot>0</slot>
<function>0</function>
<product id='0x10c9'>82576 Gigabit Network Connection</product>
<vendor id='0x8086'>Intel Corporation</vendor>
<iommuGroup number='7'>
<address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
</iommuGroup>
</capability>
</device>
Note
If there are multiple endpoints in the IOMMU group and not all of them are assigned to the guest, you will need to manually detach the other endpoint(s) from the host by running the following command before you start the guest:
$ virsh nodedev-detach pci_0000_00_19_1
Refer to the
in Section 10.1.1, “ Assigning a PCI device with virsh”
for more information on IOMMU groups.
2. Ad d t h e d evice
Use the PCI identifier output from the vi rsh no d ed ev command as the value for the --
host-device parameter.
virt-install \
--name=guest1-rhel6-64 \
--disk path=/var/lib/libvirt/images/guest1-rhel6-64.img,size=8 \
--nonsparse --graphics spice \
--vcpus=2 --ram=2048 \
--location=http://example1.com/installation_tree/RHEL6.0-Serverx86_64/os \
--nonetworks \
--os-type=linux \
--os-variant=rhel6
--host-device=pci_0000_01_00_0
3. Co mp let e t h e in st allat io n
Complete the guest installation. The PCI device should be attached to the guest.
76
Chapt er 1 0 . G uest virt ual machine device configurat ion
10.1.4 . Det aching an assigned PCI device
When a host PCI device has been assigned to a guest machine, the host can no longer use the device. Read this section to learn how to detach the device from the guest with vi rsh or virt -
man ag er so it is available for host use.
Pro ced u re 10.6 . Det ach in g a PCI d evice f ro m a g u est wit h virsh
1. Det ach t h e d evice
Use the following command to detach the PCI device from the guest by removing it in the guest's XML file:
# virsh detach-device name_of_guest file.xml
2. Re- at t ach t h e d evice t o t h e h o st ( o p t io n al)
If the device is in managed mode, skip this step. The device will be returned to the host automatically.
If the device is not using managed mode, use the following command to re-attach the PCI device to the host machine:
# virsh nodedev-reattach device
For example, to re-attach the pci _0 0 0 0 _0 1_0 0 _0 device to the host: virsh nodedev-reattach pci_0000_01_00_0
The device is now available for host use.
Pro ced u re 10.7. Det ach in g a PCI Device f ro m a g u est wit h virt - man ag er
1. O p en t h e virt u al h ard ware d et ails screen
In virt - man ag er, double-click on the virtual machine that contains the device. Select the
Sho w vi rtual hard ware d etai l s button to display a list of virtual hardware.
Fig u re 10.4 . T h e virt u al h ard ware d et ails b u t t o n
2. Select an d remo ve t h e d evice
Select the PCI device to be detached from the list of virtual devices in the left panel.
77
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 10.5. Select in g t h e PCI d evice t o b e d et ach ed
Click the R emo ve button to confirm. The device is now available for host use.
10.1.5. Creat ing PCI bridges
Peripheral Component Interconnects (PCI) bridges are used to attach to devices such as network cards, modems and sound cards. Just like their physical counterparts, virtual devices can also be attached to a PCI Bridge. In the past, only 31 PCI devices could be added to any guest virtual machine. Now, when a 31st PCI device is added, a PCI bridge is automatically placed in the 31st slot moving the additional PCI device to the PCI bridge. Each PCI bridge has 31 slots for 31 additional devices, all of which can be bridges. In this manner, over 900 devices can be available for guest virtual machines.
Note
This action cannot be performed when the guest virtual machine is running. You must add the
PCI device on a guest virtual machine that is shutdown.
10.1.6. PCI passt hrough
A PCI network device (specified by the <so urce> element) is directly assigned to the guest using generic device passthrough, after first optionally setting the device's MAC address to the configured value, and associating the device with an 802.1Qbh capable switch using an optionally specified
<vi rtual po rt> element (see the examples of virtualport given above for type='direct' network
78
Chapt er 1 0 . G uest virt ual machine device configurat ion
devices). Due to limitations in standard single-port PCI ethernet card driver design - only SR-IOV
(Single Root I/O Virtualization) virtual function (VF) devices can be assigned in this manner; to assign a standard single-port PCI or PCIe Ethernet card to a guest, use the traditional <ho std ev> device definition.
To use VFIO device assignment rather than traditional/legacy KVM device assignment (VFIO is a new method of device assignment that is compatible with UEFI Secure Boot), a <type= ' ho std ev' > interface can have an optional driver sub-element with a name attribute set to "vfio". To use legacy
KVM device assignment you can set name to "kvm" (or simply omit the <d ri ver> element, since
<d ri ver= ' kvm' > is currently the default).
Note
Intelligent passthrough of network devices is very similar to the functionality of a standard
<ho std ev> device, the difference being that this method allows specifying a MAC address and <vi rtual po rt> for the passed-through device. If these capabilities are not required, if you have a standard single-port PCI, PCIe, or USB network card that does not support SR-IOV
(and hence would anyway lose the configured MAC address during reset after being assigned to the guest domain), or if you are using a version of libvirt older than 0.9.11, you should use standard <ho std ev> to assign the device to the guest instead of <i nterface
type= ' ho std ev' />.
<devices>
<interface type= 'hostdev' >
<driver name= 'vfio' />
<source>
<address type= 'pci' domain= '0x0000' bus= '0x00' slot= '0x07' function= '0x0' />
</source>
<mac address= '52:54:00:6d:90:02' >
<virtualport type= '802.1Qbh' >
<parameters profileid= 'finance' />
</virtualport>
</interface>
</devices>
Fig u re 10.6 . XML examp le f o r PCI d evice assig n men t
10.1.7. Configuring PCI assignment (passt hrough) wit h SR-IOV devices
This section is for SR-IOV devices only. SR-IOV network cards provide multiple Virtual Functions (VFs) that can each be individually assigned to a guest virtual machines using PCI device assignment.
Once assigned, each will behave as a full physical network device. This permits many guest virtual machines to gain the performance advantage of direct PCI device assignment, while only using a single slot on the host physical machine.
These VFs can be assigned to guest virtual machines in the traditional manner using the element
<ho std ev>, but as SR-IOV VF network devices do not have permanent unique MAC addresses, it causes issues where the guest virtual machine's network settings would have to be re-configured
79
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
each time the host physical machine is rebooted. To remedy this, you would need to set the MAC address prior to assigning the VF to the host physical machine and you would need to set this each and every time the guest virtual machine boots. In order to assign this MAC address as well as other
Pro ced u re 10.8. Co n f ig u rin g MAC ad d resses, vLAN, an d virt u al p o rt s f o r assig n in g PCI d evices o n SR- IO V
It is important to note that the <ho std ev> element cannot be used for function-specific items like
MAC address assignment, vLAN tag ID assignment, or virtual port assignment because the <mac>,
<vl an>, and <vi rtual po rt> elements are not valid children for <ho std ev>. As they are valid for
<i nterface>, support for a new interface type was added (<i nterface type= ' ho std ev' >).
This new interface device type behaves as a hybrid of an <i nterface> and <ho std ev>. Thus, before assigning the PCI device to the guest virtual machine, libvirt initializes the network-specific hardware/switch that is indicated (such as setting the MAC address, setting a vLAN tag, and/or associating with an 802.1Qbh switch) in the guest virtual machine's XML configuration file. For information on setting the vLAN tag, refer to
Section 19.12, “ Setting vLAN tags”
.
1. Sh u t d o wn t h e g u est virt u al mach in e
# vi rsh shutd o wn guestVM
2. G at h er in f o rmat io n
In order to use <i nterface type= ' ho std ev' >, you must have an SR-IOV-capable network card, host physical machine hardware that supports either the Intel VT-d or AMD
IOMMU extensions, and you must know the PCI address of the VF that you wish to assign.
3. O p en t h e XML f ile f o r ed it in g
Run the # vi rsh save-i mag e-ed i t command to open the XML file for editing (refer to
Section 15.8.10, “ Edit Domain XML configuration files”
for more information). As you would want to restore the guest virtual machine to its former running state, the --runni ng would be used in this case. The name of the configuration file in this example is guestVM.xml, as the name of the guest virtual machine is guestVM.
# vi rsh save-i mag e-ed i t guestVM.xml --runni ng
The guestVM.xml opens in your default editor.
4. Ed it t h e XML f ile
Update the configuration file (guestVM.xml) to have a <d evi ces> entry similar to the following:
80
<devices>
...
<interface type= 'hostdev' managed= 'yes' >
Chapt er 1 0 . G uest virt ual machine device configurat ion
<source>
<address type= 'pci' domain= '0x0' bus= '0x00' slot= '0x07' function= '0x0' /> <!--these values can be decimal as well-->
</source>
<mac address= '52:54:00:6d:90:02' />
<!--sets the mac address-->
<virtualport type= '802.1Qbh' >
<!--sets the virtual port for the 802.1Qbh switch-->
<parameters profileid= 'finance' />
</virtualport>
<vlan>
<!--sets the vlan tag-->
<tag id= '42' />
</vlan>
</interface>
...
</devices>
Fig u re 10.7. Samp le d o main XML f o r h o st d ev in t erf ace t yp e
Note that if you do not provide a MAC address, one will be automatically generated, just as with any other type of interface device. Also, the <vi rtual po rt> element is only used if you are connecting to an 802.11Qgh hardware switch (802.11Qbg (a.k.a. "VEPA") switches are currently not supported.
5. Re- st art t h e g u est virt u al mach in e
Run the vi rsh start command to restart the guest virtual machine you shutdown in the first step (example uses guestVM as the guest virtual machine's domain name). Refer to
Section 15.8.1, “ Starting a defined domain” for more information.
# vi rsh start guestVM
When the guest virtual machine starts, it sees the network device provided to it by the physical host machine's adapter, with the configured MAC address. This MAC address will remain unchanged across guest virtual machine and host physical machine reboots.
10.1.8. Set t ing PCI device assignment from a pool of SR-IOV virt ual funct ions
Hard coding the PCI addresses of a particular Virtual Functions (VFs) into a guest's configuration has two serious limitations:
The specified VF must be available any time the guest virtual machine is started, implying that the administrator must permanently assign each VF to a single guest virtual machine (or modify the configuration file for every guest virtual machine to specify a currently unused VF's PCI address each time every guest virtual machine is started).
If the guest vitual machine is moved to another host physical machine, that host physical machine must have exactly the same hardware in the same location on the PCI bus (or, again, the guest vitual machine configuration must be modified prior to start).
It is possible to avoid both of these problems by creating a libvirt network with a device pool containing all the VFs of an SR-IOV device. Once that is done you would configure the guest virtual machine to reference this network. Each time the guest is started, a single VF will be allocated from
81
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
the pool and assigned to the guest virtual machine. When the guest virtual machine is stopped, the
VF will be returned to the pool for use by another guest virtual machine.
Pro ced u re 10.9 . Creat in g a d evice p o o l
1. Sh u t d o wn t h e g u est virt u al mach in e
guestVM.
# vi rsh shutd o wn guestVM
2. Creat e a co n f ig u rat io n f ile
Using your editor of chocice create an XML file (named passthrough.xml, for example) in the
/tmp directory. Make sure to replace pf d ev= ' eth3' with the netdev name of your own SR-
IOV device's PF
The following is an example network definition that will make available a pool of all VFs for the SR-IOV adapter with its physical function (PF) at "eth3' on the host physical machine:
<network>
<name> passthrough </name>
<!--This is the name of the file you created-->
<forward mode= 'hostdev' managed= 'yes' >
<pf dev= 'myNetDevName' />
<!--Use the netdev name of your SR-IOV devices PF here-->
</forward>
</network>
82
Fig u re 10.8. Samp le n et wo rk d ef in it io n d o main XML
3. Lo ad t h e n ew XML f ile
Run the following command, replacing /tmp/passthrough.xml, with the name and location of your XML file you created in the previous step:
# vi rsh net-d efi ne /tmp/passthrough.xml
4. Rest art in g t h e g u est
Run the following replacing passthrough.xml, with the name of your XML file you created in the previous step:
# vi rsh net-auto start passthrough # vi rsh net-start passthrough
5. Re- st art t h e g u est virt u al mach in e
Chapt er 1 0 . G uest virt ual machine device configurat ion
Run the vi rsh start command to restart the guest virtual machine you shutdown in the first step (example uses guestVM as the guest virtual machine's domain name). Refer to
Section 15.8.1, “ Starting a defined domain” for more information.
# vi rsh start guestVM
6. In it iat in g p asst h ro u g h f o r d evices
Although only a single device is shown, libvirt will automatically derive the list of all VFs associated with that PF the first time a guest virtual machine is started with an interface definition in its domain XML like the following:
<interface type= 'network' >
<source network= 'passthrough' >
</interface>
Fig u re 10.9 . Samp le d o main XML f o r in t erf ace n et wo rk d ef in it io n
7. Verif icat io n
You can verify this by running vi rsh net-d umpxml passthrough command after starting the first guest that uses the network; you will get output similar to the following:
<network connections= '1' >
<name> passthrough </name>
<uuid> a6b49429-d353-d7ad-3185-4451cc786437 </uuid>
<forward mode= 'hostdev' managed= 'yes' >
<pf dev= 'eth3' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x10' function= '0x1' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x10' function= '0x3' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x10' function= '0x5' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x10' function= '0x7' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x11' function= '0x1' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x11' function= '0x3' />
<address type= 'pci' domain= '0x0000' bus= '0x02' slot= '0x11' function= '0x5' />
83
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
</forward>
</network>
Fig u re 10.10. XML d u mp f ile passthrough co n t en t s
10.2. USB devices
This section gives the commands required for handling USB devices.
10.2.1. Assigning USB devices t o guest virt ual machines
Most devices such as web cameras, card readers, keyboards, mice etc., are connected to a computer using a USB port and cable. There are two ways to pass such devices to a guest virtual machine:
Using USB passthrough - this requires the device to be physically connected to the host physical machine that is hosting the guest virtual machine. SPICE is not needed in this case. USB devices on the host can be passed to the guest using the command line or virt - man ag er. Refer to
Section 16.3.1, “ Attaching USB devices to a guest virtual machine”
for virt man ag er directions.
Note
virt - man ag er should not be used for hot plugging or hot unplugging devices. If you want
Using USB re-direction - USB re-direction is best used in cases where there is a host physical machine that is running in a data center. The user connects to his/her guest virtual machine from a local machine or thin client. On this local machine there is a SPICE client. The user can attach any USB device to the thin client and the SPICE client will redirect the device to the host physical machine on the data center so it can be used by the guest virtual machine that is running on the
thin client. For instructions on USB re-direction using the virt-manager, refer to Section 16.3.1,
“ Attaching USB devices to a guest virtual machine” It should be noted that USB redirection is not
possible using the TCP protocol (Refer to BZ #1085318 ).
10.2.2. Set t ing a limit on USB device redirect ion
To filter out certain devices from redirection, pass the filter property to -device usb-redir. The filter property takes a string consisting of filter rules, the format for a rule is:
<cl ass>: <vend o r>: <pro d uct>: <versi o n>: <al l o w>
Use the value -1 to designate it to accept any value for a particular field. You may use multiple rules on the same command line using | as a separator.
Important
If a device matches none of the rule filters, redirecting it will not be allowed!
84
Chapt er 1 0 . G uest virt ual machine device configurat ion
Examp le 10.1. An examp le o f limit in g red irect io n wit h a win d o ws g u est virt u al
1. Prepare a Windows 7 guest virtual machine.
2. Add the following code excerpt to the guest virtual machine's' domain xml file:
<redirdev bus='usb' type='spicevmc'>
<alias name='redir0'/>
<address type='usb' bus='0' port='3'/>
</redirdev>
<redirfilter>
<usbdev class='0x08' vendor='0x1234' product='0xBEEF' version='2.0' allow='yes'/>
<usbdev class='-1' vendor='-1' product='-1' version='-1' allow='no'/>
</redirfilter>
3. Start the guest virtual machine and confirm the setting changes by running the following:
#ps -ef | g rep $g uest_name
-d evi ce usb-red i r,chard ev= charred i r0 ,i d = red i r0 ,/
fi l ter= 0 x0 8: 0 x1234 : 0 xBEEF: 0 x0 20 0 : 1| -1: -1: -1: -
1: 0 ,bus= usb. 0 ,po rt= 3
4. Plug a USB device into a host physical machine, and use virt - man ag er to connect to the guest virtual machine.
5. Click Red irect USB Service in the menu, which will produce the following message:
"Some USB devices are blocked by host policy". Click O K to confirm and continue.
The filter takes effect.
6. To make sure that the filter captures properly check the USB device vendor and product, then make the following changes in the guest virtual machine's domain XML to allow for
USB redirection.
<redirfilter>
<usbdev class='0x08' vendor='0x0951' product='0x1625' version='2.0' allow='yes'/>
<usbdev allow='no'/>
</redirfilter>
7. Restart the guest virtual machine, then use virt - viewer to connect to the guest virtual machine. The USB device will now redirect traffic to the guest virtual machine.
10.3. Configuring device cont rollers
Depending on the guest virtual machine architecture, some device buses can appear more than once, with a group of virtual devices tied to a virtual controller. Normally, libvirt can automatically infer such controllers without requiring explicit XML markup, but in some cases it is better to explicitly set a virtual controller element.
85
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
...
<devices>
<controller type= 'ide' index= '0' />
<controller type= 'virtio-serial' index= '0' ports= '16' vectors= '4' />
<controller type= 'virtio-serial' index= '1' >
<address type= 'pci' domain= '0x0000' bus= '0x00' slot= '0x0a' function= '0x0' />
</controller>
...
</devices>
...
Fig u re 10.11. Do main XML examp le f o r virt u al co n t ro llers
Each controller has a mandatory attribute <co ntro l l er type>, which must be one of: ide fdc scsi sata usb ccid virtio-serial pci
The <co ntro l l er> element has a mandatory attribute <co ntro l l er i nd ex> which is the decimal integer describing in which order the bus controller is encountered (for use in controller attributes of
<ad d ress> elements). When <co ntro l l er type = ' vi rti o -seri al ' > there are two additional optional attributes (named po rts and vecto rs), which control how many devices can be connected through the controller.
When <co ntro l l er type = ' scsi ' >, there is an optional attribute mo d el model, which can have the following values: auto buslogic ibmvscsi lsilogic lsisas1068 lsisas1078 virtio-scsi
86
Chapt er 1 0 . G uest virt ual machine device configurat ion
vmpvscsi
When <co ntro l l er type = ' usb' >, there is an optional attribute mo d el model, which can have the following values: piix3-uhci piix4-uhci ehci ich9-ehci1 ich9-uhci1 ich9-uhci2 ich9-uhci3 vt82c686b-uhci pci-ohci nec-xhci
Note
If the USB bus needs to be explicitly disabled for the guest virtual machine,
<mo d el = ' no ne' > may be used. .
For controllers that are themselves devices on a PCI or USB bus, an optional sub-element
<ad d ress> can specify the exact relationship of the controller to its master bus, with semantics as
shown in Section 10.4, “ Setting addresses for devices”
.
An optional sub-element <d ri ver> can specify the driver specific options. Currently it only supports attribute queues, which specifies the number of queues for the controller. For best performance, it's recommended to specify a value matching the number of vCPUs.
USB companion controllers have an optional sub-element <master> to specify the exact relationship of the companion to its master controller. A companion controller is on the same bus as its master, so the companion i nd ex value should be equal.
An example XML which can be used is as follows:
...
<devices>
<controller type= 'usb' index= '0' model= 'ich9-ehci1' >
<address type= 'pci' domain= '0' bus= '0' slot= '4' function= '7' />
</controller>
<controller type= 'usb' index= '0' model= 'ich9-uhci1' >
<master startport= '0' />
<address type= 'pci' domain= '0' bus= '0' slot= '4' function= '0' multifunction= 'on' />
87
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
</controller>
...
</devices>
...
Fig u re 10.12. Do main XML examp le f o r USB co n t ro llers
PCI controllers have an optional mo d el attribute with the following possible values: pci-root pcie-root pci-bridge dmi-to-pci-bridge
The root controllers (pci -ro o t and pci e-ro o t) have an optional pci ho l e6 4 element specifying how big (in kilobytes, or in the unit specified by pci ho l e6 4 's uni t attribute) the 64-bit PCI hole should be. Some guest virtual machines (such as Windows XP or Windows Server 2003) may cause a crash, unless uni t is disabled (set to 0 uni t= ' 0 ' ).
For machine types which provide an implicit PCI bus, the pci-root controller with i nd ex= ' 0 ' is auto-added and required to use PCI devices. pci-root has no address. PCI bridges are auto-added if there are too many devices to fit on the one bus provided by mo d el = ' pci -ro o t' , or a PCI bus number greater than zero was specified. PCI bridges can also be specified manually, but their addresses should only refer to PCI buses provided by already specified PCI controllers. Leaving gaps in the PCI controller indexes might lead to an invalid configuration. The following XML example can be added to the <d evi ces> section:
...
<devices>
<controller type= 'pci' index= '0' model= 'pci-root' />
<controller type= 'pci' index= '1' model= 'pci-bridge' >
<address type= 'pci' domain= '0' bus= '0' slot= '5' function= '0' multifunction= 'off' />
</controller>
</devices>
...
Fig u re 10.13. Do main XML examp le f o r PCI b rid g e
For machine types which provide an implicit PCI Express (PCIe) bus (for example, the machine types based on the Q35 chipset), the pcie-root controller with i nd ex= ' 0 ' is auto-added to the domain's configuration. pcie-root has also no address, but provides 31 slots (numbered 1-31) and can only be used to attach PCIe devices. In order to connect standard PCI devices on a system which has a pcieroot controller, a pci controller with mo d el = ' d mi -to -pci -bri d g e' is automatically added. A dmi-to-pci-bridge controller plugs into a PCIe slot (as provided by pcie-root), and itself provides 31
88
Chapt er 1 0 . G uest virt ual machine device configurat ion
standard PCI slots (which are not hot-pluggable). In order to have hot-pluggable PCI slots in the guest system, a pci-bridge controller will also be automatically created and connected to one of the slots of the auto-created dmi-to-pci-bridge controller; all guest devices with PCI addresses that are auto-determined by libvirt will be placed on this pci-bridge device.
...
<devices>
<controller type= 'pci' index= '0' model= 'pcie-root' />
<controller type= 'pci' index= '1' model= 'dmi-to-pci-bridge' >
<address type= 'pci' domain= '0' bus= '0' slot= '0xe' function= '0' />
</controller>
<controller type= 'pci' index= '2' model= 'pci-bridge' >
<address type= 'pci' domain= '0' bus= '1' slot= '1' function= '0' />
</controller>
</devices>
...
Fig u re 10.14 . Do main XML examp le f o r PCIe ( PCI exp ress)
10.4 . Set t ing addresses for devices
Many devices have an optional <ad d ress> sub-element which is used to describe where the device is placed on the virtual bus presented to the guest virtual machine. If an address (or any optional attribute within an address) is omitted on input, libvirt will generate an appropriate address; but an
PCI device assignment” for domain XML device examples including an <ad d ress> element.
Every address has a mandatory attribute type that describes which bus the device is on. The choice of which address to use for a given device is constrained in part by the device and the architecture of the guest virtual machine. For example, a <d i sk> device uses type= ' d ri ve' , while a <co nso l e> device would use type= ' pci ' on i686 or x86_64 guest virtual machine architectures. Each address type has further optional attributes that control where on the bus the device will be placed as described in the table:
T ab le 10.1. Su p p o rt ed d evice ad d ress t yp es
Ad d ress t yp e Descrip t io n
89
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Ad d ress t yp e
type='pci' type='drive' type='virtio-serial' type='ccid' type='usb' type='isa'
Descrip t io n
PCI addresses have the following additional attributes: domain (a 2-byte hex integer, not currently used by qemu) bus (a hex value between 0 and 0xff, inclusive) slot (a hex value between 0x0 and 0x1f, inclusive) function (a value between 0 and 7, inclusive) multifunction controls turning on the multifunction bit for a particular slot/function in the PCI control register By default it is set to 'off', but should be set to 'on' for function 0 of a slot that will have multiple functions used.
Drive addresses have the following additional attributes: controller (a 2-digit controller number) bus (a 2-digit bus number target (a 2-digit bus number) unit (a 2-digit unit number on the bus)
Each virtio-serial address has the following additional attributes: controller (a 2-digit controller number) bus (a 2-digit bus number) slot (a 2-digit slot within the bus)
A CCID address, for smart-cards, has the following additional attributes: bus (a 2-digit bus number) slot attribute (a 2-digit slot within the bus)
USB addresses have the following additional attributes: bus (a hex value between 0 and 0xfff, inclusive) port (a dotted notation of up to four octets, such as 1.2 or 2.1.3.1)
ISA addresses have the following additional attributes: iobase irq
10.5. Managing st orage cont rollers in a guest virt ual machine
90
Chapt er 1 0 . G uest virt ual machine device configurat ion
Starting from Red Hat Enterprise Linux 6.4, it is supported to add SCSI and virtio-SCSI devices to guest virtual machines that are running Red Hat Enterprise Linux 6.4 or later. Unlike virtio disks, SCSI devices require the presence of a controller in the guest virtual machine. Virtio-SCSI provides the ability to connect directly to SCSI LUNs and significantly improves scalability compared to virtio-blk.
The advantage of virtio-SCSI is that it is capable of handling hundreds of devices compared to virtioblk which can only handle 28 devices and exhausts PCI slots. Virtio-SCSI is now capable of inheriting the feature set of the target device with the ability to: attach a virtual hard drive or CD through the virtio-scsi controller, pass-through a physical SCSI device from the host to the guest via the QEMU scsi-block device, and allow the usage of hundreds of devices per guest; an improvement from the 28-device limit of virtio-blk.
This section details the necessary steps to create a virtual SCSI controller (also known as "Host Bus
Adapter", or HBA) and to add SCSI storage to the guest virtual machine.
Pro ced u re 10.10. Creat in g a virt u al SCSI co n t ro ller
1. Display the configuration of the guest virtual machine (G uest1) and look for a pre-existing
SCSI controller:
# virsh dumpxml Guest1 | grep controller.*scsi
If a device controller is present, the command will output one or more lines similar to the following:
<controller type='scsi' model='virtio-scsi' index='0'/>
2. If the previous step did not show a device controller, create the description for one in a new file and add it to the virtual machine, using the following steps: a. Create the device controller by writing a <co ntro l l er> element in a new file and save this file with an XML extension. vi rti o -scsi -co ntro l l er. xml , for example.
<controller type='scsi' model='virtio-scsi'/> b. Associate the device controller you just created in vi rti o -scsi -co ntro l l er. xml with your guest virtual machine (Guest1, for example):
# virsh attach-device --config Guest1 ~/virtio-scsicontroller.xml
In this example the --co nfi g option behaves the same as it does for disks. Refer to
Procedure 14.2, “ Adding physical block devices to guests” for more information.
3. Add a new SCSI disk or CD-ROM. The new disk can be added using the methods in sections
Section 14.3.1, “ Adding file based storage to a guest”
device name that starts with sd.
# virsh attach-disk Guest1 /var/lib/libvirt/images/FileName.img sdb
--cache none
Depending on the version of the driver in the guest virtual machine, the new disk may not be
91
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
detected immediately by a running guest virtual machine. Follow the steps in the Red Hat
Enterprise Linux Storage Administration Guide.
10.6. Random number generat or (RNG) device
virtio-rng is a virtual RNG (random number generator) device that feeds RNG data to the guest virtual machine's operating system, thereby providing fresh entropy for guest virtual machines on request.
Using an RNG is particularly useful when a device such as a keyboard, mouse and other inputs are not enough to generate sufficient entropy on the guest virtual machine. The virtio-rng device is available for both Red Hat Enterprise Linux and Windows guest virtual machines. Refer to the
for instructions on installing the Windows requirements. Unless noted, the following descriptions are for both Red Hat Enterprise Linux and Windows guest virtual machines.
When virtio-rng is enabled on a Linux guest virtual machine, a chardev is created in the guest virtual machine at the location /d ev/hwrng . This chardev can then be opened and read to fetch entropy from the host physical machine. In order for guest virtual machines' applications to benefit from using randomness from the virtio-rng device transparently, the input from /d ev/hwrng must be relayed to the kernel entropy pool in the guest virtual machine. This can be accomplished if the information in this location is coupled with the rgnd daemon (contained within the rng-tools).
This coupling results in the entropy to be routed to the guest virtual machine's /d ev/rand o m file.
The process is done manually in Red Hat Enterprise Linux 6 guest virtual machines.
Red Hat Enterprise Linux 6 guest virtual machines are coupled by running the following command:
# rngd -b -r /dev/hwrng -o /dev/random
For more assistance, run the man rng d command for an explanation of the command options
Note
Windows guest virtual machines require the driver viorng to be installed. Once installed, the virtual RNG device will work using the CNG (crypto next generation) API provided by Microsoft.
Once the driver is installed, the virtrng device appears in the list of RNG providers.
Pro ced u re 10.11. Imp lemen t in g virt io - rn g wit h t h e co mman d lin e t o o ls
1. Shut down the guest virtual machine.
2. In a terminal window, using the vi rsh ed i t domain-name command, open the XML file for the desired guest virtual machine.
3. Edit the <d evi ces> element to include the following:
...
<devices>
<rng model= 'virtio' >
<rate period= "2000" bytes= "1234" />
<backend model= 'random' > /dev/random </backend>
92
Chapt er 1 0 . G uest virt ual machine device configurat ion
<source mode= 'bind' service= '1234' >
<source mode= 'connect' host= '192.0.2.1' service= '1234' >
</backend>
</rng>
</devices>
...
93
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 11. QEMU-img and QEMU guest agent
This chapter contain useful hints and tips for using the qemu-img package with guest virtual machines. If you are looking for information on QEMU trace events and arguments, refer to the
README file located here: /usr/share/d o c/q emu-*/R EAD ME. systemtap.
11.1. Using qemu-img
The q emu-i mg command line tool is used for formatting, modifying and verifying various file systems used by KVM. q emu-i mg options and usages are listed below.
Ch eck
Perform a consistency check on the disk image filename.
# qemu-img check [-f format] filename
Note
Only the q co w2 and vd i formats support consistency checks.
Co mmit
Commit any changes recorded in the specified file (filename) to the file's base image with the q emu-
i mg co mmi t command. Optionally, specify the file's format type (fmt).
# qemu-img commit [-f fmt] [-t cache] filename
Co n vert
The convert option is used to convert one recognized image format to another image format.
Command format:
# qemu-img convert [-c] [-p] [-f fmt] [-t cache] [-O output_fmt] [-o
options] [-S sparse_size] filename output_filename
The -p parameter shows the progress of the command (optional and not for every command) and -S flag allows for the creation of a sparse file, which is included within the disk image. Sparse files in all purposes function like a standard file, except that the physical blocks that only contain zeros (i.e., nothing). When the Operating System sees this file, it treats it as it exists and takes up actual disk space, even though in reality it doesn't take any. This is particularly helpful when creating a disk for a guest virtual machine as this gives the appearance that the disk has taken much more disk space than it has. For example, if you set -S to 50Gb on a disk image that is 10Gb, then your 10Gb of disk space will appear to be 60Gb in size even though only 10Gb is actually being used.
Convert the disk image filename to disk image output_filename using format output_format.
The disk image can be optionally compressed with the -c option, or encrypted with the -o option by setting -o encrypti o n. Note that the options available with the -o parameter differ with the selected format.
94
Chapt er 1 1 . Q EMU- img and Q EMU guest agent
Only the q co w2 format supports encryption or compression. q co w2 encryption uses the AES format with secure 128-bit keys. q co w2 compression is read-only, so if a compressed sector is converted from q co w2 format, it is written to the new format as uncompressed data.
Image conversion is also useful to get a smaller image when using a format which can grow, such as
q co w or co w. The empty sectors are detected and suppressed from the destination image.
Creat e
Create the new disk image filename of size size and format format.
# qemu-img create [-f format] [-o options] filename [size]
If a base image is specified with -o backi ng _fi l e= filename, the image will only record differences between itself and the base image. The backing file will not be modified unless you use the co mmi t command. No size needs to be specified in this case.
In f o
The i nfo parameter displays information about a disk image filename. The format for the i nfo option is as follows:
# qemu-img info [-f format] filename
This command is often used to discover the size reserved on disk which can be different from the displayed size. If snapshots are stored in the disk image, they are displayed also. This command will show for example, how much space is being taken by a q co w2 image on a block device. This is done by running the q emu-i mg . You can check that the image in use is the one that matches the output of the q emu-i mg i nfo command with the q emu-i mg check command. Refer to
Section 11.1, “ Using qemu-img” .
# qemu-img info /dev/vg-90.100-sluo/lv-90-100-sluo image: /dev/vg-90.100-sluo/lv-90-100-sluo file format: qcow2 virtual size: 20G (21474836480 bytes) disk size: 0 cluster_size: 65536
Reb ase
Changes the backing file of an image.
# qemu-img rebase [-f fmt] [-t cache] [-p] [-u] -b backing_file [-F
backing_fmt] filename
The backing file is changed to backing_file and (if the format of filename supports the feature), the backing file format is changed to backing_format.
Note
Only the q co w2 format supports changing the backing file (rebase).
There are two different modes in which rebase can operate: Saf e and Un saf e.
95
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Saf e mo d e is used by default and performs a real rebase operation. The new backing file may differ from the old one and the q emu-i mg rebase command will take care of keeping the guest virtual machine-visible content of filename unchanged. In order to achieve this, any clusters that differ between backing_file and old backing file of filename are merged into filename before making any changes to the backing file.
Note that safe mode is an expensive operation, comparable to converting an image. The old backing file is required for it to complete successfully.
Un saf e mo d e is used if the -u option is passed to q emu-i mg rebase. In this mode, only the backing file name and format of filename is changed, without any checks taking place on the file contents. Make sure the new backing file is specified correctly or the guest-visible content of the image will be corrupted.
This mode is useful for renaming or moving the backing file. It can be used without an accessible old backing file. For instance, it can be used to fix an image whose backing file has already been moved or renamed.
Resiz e
Change the disk image filename as if it had been created with size size. Only images in raw format can be resized regardless of version. Red Hat Enterprise Linux 6.1 and later adds the ability to grow (but not shrink) images in q co w2 format.
Use the following to set the size of the disk image filename to size bytes:
# qemu-img resize filename size
You can also resize relative to the current size of the disk image. To give a size relative to the current size, prefix the number of bytes with + to grow, or - to reduce the size of the disk image by that number of bytes. Adding a unit suffix allows you to set the image size in kilobytes (K), megabytes (M), gigabytes (G) or terabytes (T).
# qemu-img resize filename [+|-]size[K|M|G|T]
Warning
Before using this command to shrink a disk image, you must use file system and partitioning tools inside the VM itself to reduce allocated file systems and partition sizes accordingly.
Failure to do so will result in data loss.
After using this command to grow a disk image, you must use file system and partitioning tools inside the VM to actually begin using the new space on the device.
Sn ap sh o t
List, apply, create, or delete an existing snapshot (snapshot) of an image (filename).
# qemu-img snapshot [ -l | -a snapshot | -c snapshot | -d snapshot ]
filename
96
Chapt er 1 1 . Q EMU- img and Q EMU guest agent
-l lists all snapshots associated with the specified disk image. The apply option, -a, reverts the disk image (filename) to the state of a previously saved snapshot. -c creates a snapshot (snapshot) of an image (filename). -d deletes the specified snapshot.
Su p p o rt ed f o rmat s
q emu - img is designed to convert files to one of the following formats:
raw
Raw disk image format (default). This can be the fastest file-based format. If your file system supports holes (for example in ext2 or ext3 on Linux or NTFS on Windows), then only the written sectors will reserve space. Use q emu-i mg i nfo to obtain the real size used by the image or l s -l s on Unix/Linux. Although Raw images give optimal performance, only very basic features are available with a Raw image (no snapshots etc.).
q co w2
QEMU image format, the most versatile format with the best feature set. Use it to have optional AES encryption, zlib-based compression, support of multiple VM snapshots, and smaller images, which are useful on file systems that do not support holes (non-NTFS file systems on Windows). Note that this expansive feature set comes at the cost of performance.
Although only the formats above can be used to run on a guest virtual machine or host physical machine machine, q emu - img also recognizes and supports the following formats in order to convert from them into either raw or q co w2 format. The format of an image is usually detected automatically.
In addition to converting these formats into raw or q co w2 , they can be converted back from raw or
q co w2 to the original format.
bo chs
Bochs disk image format.
cl o o p
Linux Compressed Loop image, useful only to reuse directly compressed CD-ROM images present for example in the Knoppix CD-ROMs.
co w
User Mode Linux Copy On Write image format. The co w format is included only for compatibility with previous versions. It does not work with Windows.
d mg
Mac disk image format.
nbd
Network block device.
paral l el s
Parallels virtualization disk image format.
q co w
Old QEMU image format. Only included for compatibility with older versions.
vd i
97
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Oracle VM VirtualBox hard disk image format.
vmd k
VMware compatible image format (read-write support for versions 1 and 2, and read-only support for version 3).
vpc
Windows Virtual PC disk image format. Also referred to as vhd , or Microsoft virtual hard disk image format.
vvfat
Virtual VFAT disk image format.
11.2. QEMU Guest Agent
The QEMU Guest Agent allows the host machine to issue commands to the guest operating system.
The guest operating system then responds to those commands asynchronously.
This section covers the options and commands available to the guest agent in detail. It also covers how to run the Guest Agent in the foreground, or as a daemon in the background. If you are looking for information on QEMU trace events and arguments, refer to the README file located here:
/usr/share/d o c/q emu-*/R EAD ME. systemtap.
Note that CPU hot plugging and hot unplugging are supported with the help of the QEMU guest agent on Linux and Windows guests; CPUs can be enabled or disabled while the guest is running,
thus implementing the hotplug feature and mimicking the unplug feature. Refer to Section 15.13.6,
“ Configuring virtual CPU count” for the command to use to implement this.
11.2.1. QEMU guest virt ual machine agent prot ocol
The QEMU guest virtual machine agent protocol (QEMU GA), uses the same protocol as QMP. The
QEMU GA package, qemu-guest-agent, is fully supported in Red Hat Enterprise Linux 6.5. There are some issues regarding its isa-serial/virtio-serial transport, and the following caveats have been noted:
There is no way for qemu-guest-agent to detect whether or not a client has connected to the channel.
There is no way for a client to detect whether or not qemu-guest-agent has disconnected or reconnected to the backend.
If the virtio-serial device resets and qemu-guest-agent has not connected to the channel as a result,
(generally caused by a reboot or hotplug), data from the client will be dropped.
If qemu-guest-agent has connected to the channel following a virtio-serial device reset, data from the client will be queued (and eventually throttled if available buffers are exhausted), regardless of whether or not qemu-guest-agent is still running/connected.
QEMU GA uses the guest virtual machine-sync or guest virtual machine-sync-delimited command to address the problem of re-synchronizing the channel after re-connection or client-side timeouts.
These are described below.
11.2.2. guest -sync
98
Chapt er 1 1 . Q EMU- img and Q EMU guest agent
The guest-sync request/response exchange is simple. The client provides a unique numerical token, the agent sends it back in a response:
> { "execute": "guest-sync", "arguments": { "id": 123456 } }
< { "return": 123456}
A successful exchange guarantees that the channel is now in sync and no unexpected data/responses will be sent. Note that for the reasons mentioned above there's no guarantee this request will be answered, so a client should implement a timeout and re-issue this periodically until a response is received for the most recent request.
This alone does not handle synchronization issues in all cases. For example, if qemu-guest-agent's parser previously received a partial request from a previous client connection, subsequent attempts to issue the guest-sync request can be misconstrued as being part of the previous partial request.
Eventually qemu-guest-agent will hit it's recursion or token size limit and flush its parser state, at which point it will begin processing the backlog of requests, but there's no guarantee this will occur before the channel is throttled due to exhausting all available buffers. Thus, there is a potential for a deadlock situation occurring for certain instances.
To avoid this, qemu-guest-agent/QEMU's JSON parser has special handling for the 0xFF byte, which is an invalid UTF-8 character. Client requests should precede the guest-sync request to ensure that
qemu-guest-agent flushes it's parser state as soon as possible. As long as all clients abide by this, the deadlock state should be reliably avoidable.
For more information see the qemu-guest-agent wiki page on wiki.qemu.org
. Additional documentation can also be found in the /usr/share/d o c/q emu-*/R EAD ME. systemtap README file.
11.2.3. guest -sync-delimit ed
If qemu-guest-agent attempts to communicate with a client, and the client receives a partial response from a previous qemu-guest-agent instance, the client might misconstrue responses to guest-sync as being part of this previous request. For client implementations that treat newlines as a delimiter for
qemu-guest-agent responses, use g uest-sync-d el i mi ted .
Even in some cases where there are JSON stream-based implementations that do not rely on newline delimiters, it may be considered invasive to implement a client's response/JSON handling, as it is the same deadlock scenario described previously. Using the g uest-sync-d el i mi ted on the client, tells QEMU GA to place the same 0xFF character in front of the response, thereby preventing confusion.
> { "execute": "guest-sync-delimited", "arguments": { "id": 123456 } }
< { "return": 123456}
Actual hex values sent:
> 7b 27 65 78 65 63 75 74 65 27 3a 27 67 75 65 73 74 2d 73 79 6e 63 2d 64
65
6c 69 6d 69 74 65 64 27 2c 27 61 72 67 75 6d 65 6e 74 73 27 3a 7b 27 69
64
27 3a 31 32 33 34 35 36 7d 7d 0a
< ff 7b 22 72 65 74 75 72 6e 22 3a 20 31 32 33 34 35 36 7d 0a
As stated above, the request should also be preceded with a 0xFF to flush qemu-guest-agent's parser state.
99
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
11.2.4 . Creat ing a guest virt ual machine disk backup
In Red Hat Enterprise Linux 6.4, the QEMU GA provided protection against the corruption of Linux guest virtual machines. Just before issuing the snapshot request, or creating a backup copy of the disk, the management stack (libvirt) sends a g uest-fsfreeze-freeze QMP command to the QEMU
GA via the virtio-serial port. This command causes the guest agent to freeze all of the guest virtual machine's filesystems, via the FIFR EEZE i o ctl () kernel function. This i o ctl () function is implemented by the Linux kernel in the guest virtual machine. The function flushes the filesystem cache in the guest virtual machine's kernel, brings the filesystem into a consistent state, and denies all userspace threads write access to the filesystem.
Only after the QEMU GA reported success, libvirt would proceed with the snapshot. At its completion,
libvirt sends the g uest-fsfreeze-thaw QMP command to the QEMU GA over the virtio-serial port.
This command tells the QEMU GA to issue a FIT HAW i o ctl (), which unblocks the userspace threads that were previously denied write access, and resumes normal processing. The caveat of this process is that it didnt' ensure that application-level data is in a consistent state when the virtual disk snapshot is taken. This is evident in cases where the fsck utility doesn't find any problems on the filesystem restored from a snapshot, and yet applications are not able to resume processing from the point where the snapshot was taken and userspace processes may not have written their internal buffers to files on the disk.
Improvements in Red Hat Enterprise Linux 6.5 have been made to make sure that both file and application level synchronization (flushing) is done. Guest system administrators can write and install application-specific freezing and thawing hook scripts. Before freezing the filesystems, the
QEMU GA invokes the main hook script (included in the QEMU GA package). The main hook script in turn calls individual application-specific scripts, prepared by the guest system administrators, that temporarily deactivates all guest virtual machine applications. All of these actions are done when the mode is changed to freeze.
Just before filesystems are frozen, the guest system administrator's scripts trigger the databases and other file system applications to flush their working buffers to the virtual disk and to stop accepting further client connections. The applications should bring their data files into a consistent state where resumption of processing, with the reactivated (or a freshly started) instance of the application (after restoring the virtual disk from backup) would be possible. When all scripts are done inactivating their respective applications, and the main hook script returns, QEMU GA proceeds to freeze filesystems, and the management stack takes the snapshot. Once all this is done, and it is confirmed that the snapshot is taken, the file system will resume to serve write requests. This process is called thawing.
Thawing happens in reverse order. Instructed by libvirt, QEMU GA thaws the guest virtual machine's filesystems. It then invokes individual hook scripts (via the main hook script) to resume or restart applications that had been inactivated during the freeze process.
Tips for Windows guests
Windows guest virtual machines will use QEMU GA for windows, qemu-guest-agent-win. This agent allows for VSS (Volume Shadow Copy Service) support for Windows guest virtual machines running on Red Hat Enterprise Linux. More information can be found here .
100
Chapt er 1 1 . Q EMU- img and Q EMU guest agent
Tips when using SELinux
An application specific hook script might need various SELinux permissions in order to run correctly. As is done when the script needs to connect to a socket in order to talk to a database. In general, local SELinux policies should be developed and installed for such purposes. Accessing file system nodes should work out of the box, after issuing the
resto reco n -FvvR command listed in
Table 11.1, “ QEMU guest agent package contents” in
the table row labeled /usr/l i bexec/q emu-g a/fsfreeze-ho o k. d /.
Additional information about the Freeze hook
Pro ced u re 11.1. En ab lin g t h e f reez e h o o k
1. Stop the qemu-ga service, using the following command:
# service qemu-ga stop
2. Change qemu-ga config-file to enable FSFREEZ E_HOOK_ENABLE, using the following command or you can open the file and change it manually:
# sed -i '/FSFREEZE_HOOK_ENABLE=0/ s/0/1/' /etc/sysconfig/qemu-ga
3. Start the service again wit the following command:
# service qemu-ga start
The qemu-guest-agent binary RPM includes the following files:
T ab le 11.1. Q EMU g u est ag en t p ackag e co n t en t s
File n ame
/etc/rc. d /i ni t. d /q emu-g a
/etc/sysco nfi g /q emu-g a
/usr/bi n/q emu-g a
/usr/l i bexec/q emu-g a/
/usr/l i bexec/q emu-g a/fsfreeze-ho o k
Descrip t io n
Service control script (start/stop) for the QEMU
GA.
Configuration file for the QEMU guest agent, as it is read by the /etc/rc. d /i ni t. d /q emu-
g a control script. The settings are documented in the file with shell script comments.
QEMU GA binary file.
Root directory for hook scripts.
Main hook script. No modifications are needed here.
101
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
File n ame
/usr/l i bexec/q emu-g a/fsfreezeho o k. d /
/usr/share/q emu-kvm/q emu-g a/
Descrip t io n
Directory for individual, application-specific hook scripts. The guest system administrator should copy hook scripts manually into this directory, ensure proper file mode bits for them, and then run resto reco n -FvvR on this directory.
Directory with sample scripts (for example purposes only). The scripts contained here are not executed.
The main hook script, /usr/l i bexec/q emu-g a/fsfreeze-ho o k logs its own messages, as well as the application-specific scripts' standard output and error messages, in the following log file:
/var/l o g /q emu-g a/fsfreeze-ho o k. l o g . For more information, refer to the qemu-guest-agent wiki page on wiki.qemu.org
or libvirt.org
.
11.3. Set t ing a limit on device redirect ion
To filter out certain devices from redirection, pass the filter property to -device usb-redir. The filter property takes a string consisting of filter rules, the format for a rule is:
<cl ass>: <vend o r>: <pro d uct>: <versi o n>: <al l o w>
Use the value -1 to designate it to accept any value for a particular field. You may use multiple rules on the same command line using | as a separator. Note that if a device matches none of the passed in rules, redirecting it will not be allowed!
Examp le 11.1. An examp le o f limit in g red irect io n wit h a win d o ws g u est virt u al mach in e
1. Prepare a Windows 7 guest virtual machine.
2. Add the following code excerpt to the guest virtual machine's' domain xml file:
<redirdev bus='usb' type='spicevmc'>
<alias name='redir0'/>
<address type='usb' bus='0' port='3'/>
</redirdev>
<redirfilter>
<usbdev class='0x08' vendor='0x1234' product='0xBEEF' version='2.0' allow='yes'/>
<usbdev class='-1' vendor='-1' product='-1' version='-1' allow='no'/>
</redirfilter>
3. Start the guest virtual machine and confirm the setting changes by running the following:
#ps -ef | g rep $g uest_name
-d evi ce usb-red i r,chard ev= charred i r0 ,i d = red i r0 ,/
fi l ter= 0 x0 8: 0 x1234 : 0 xBEEF: 0 x0 20 0 : 1| -1: -1: -1: -
1: 0 ,bus= usb. 0 ,po rt= 3
102
Chapt er 1 1 . Q EMU- img and Q EMU guest agent
4. Plug a USB device into a host physical machine, and use virt-viewer to connect to the guest virtual machine.
5. Click USB d evice select io n in the menu, which will produce the following message:
"Some USB devices are blocked by host policy". Click O K to confirm and continue.
The filter takes effect.
6. To make sure that the filter captures properly check the USB device vendor and product, then make the following changes in the host physical machine's domain XML to allow for
USB redirection.
<redirfilter>
<usbdev class='0x08' vendor='0x0951' product='0x1625' version='2.0' allow='yes'/>
<usbdev allow='no'/>
</redirfilter>
7. Restart the guest virtual machine, then use virt - viewer to connect to the guest virtual machine. The USB device will now redirect traffic to the guest virtual machine.
11.4 . Dynamically changing a host physical machine or a net work bridge t hat is at t ached t o a virt ual NIC
This section demonstrates how to move the vNIC of a guest virtual machine from one bridge to another while the guest virtual machine is running without compromising the guest virtual machine
1. Prepare guest virtual machine with a configuration similar to the following:
<interface type='bridge'>
<mac address='52:54:00:4a:c9:5e'/>
<source bridge='virbr0'/>
<model type='virtio'/>
</interface>
2. Prepare an XML file for interface update:
# cat br1. xml
<interface type='bridge'>
<mac address='52:54:00:4a:c9:5e'/>
<source bridge='virbr1'/>
<model type='virtio'/>
</interface>
3. Start the guest virtual machine, confirm the guest virtual machine's network functionality, and check that the guest virtual machine's vnetX is connected to the bridge you indicated.
# brctl sho w
bridge name bridge id STP enabled interfaces virbr0 8000.5254007da9f2 yes virbr0-nic
103
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
vnet0 virbr1 8000.525400682996 yes virbr1-nic
4. Update the guest virtual machine's network with the new interface parameters with the following command:
# vi rsh upd ate-d evi ce test1 br1. xml
Device updated successfully
5. On the guest virtual machine, run servi ce netwo rk restart. The guest virtual machine gets a new IP address for virbr1. Check the guest virtual machine's vnet0 is connected to the new bridge(virbr1)
# brctl sho w bridge name bridge id STP enabled interfaces virbr0 8000.5254007da9f2 yes virbr0-nic virbr1 8000.525400682996 yes virbr1-nic vnet0
104
Chapt er 1 2 . St orage concept s
Chapter 12. Storage concepts
This chapter introduces the concepts used for describing and managing storage devices. Terms such as Storage pools and Volumes are explained in the sections that follow.
12.1. St orage pools
A storage pool is a file, directory, or storage device managed by libvirt for the purpose of providing storage to guest virtual machines. The storage pool can be local or it can be shared over a network.
A storage pool is a quantity of storage set aside by an administrator, often a dedicated storage administrator, for use by guest virtual machines. Storage pools are divided into storage volumes either by the storage administrator or the system administrator, and the volumes are assigned to guest virtual machines as block devices. In short storage volumes are to partitions what storage pools are to disks. Although the storage pool is a virtual container it is limited by two factors: maximum size allowed to it by qemu-kvm and the size of the disk on the host physical machine.
Storage pools may not exceed the size of the disk on the host physical machine. The maximum sizes are as follows: virtio-blk = 2^63 bytes or 8 Exabytes(using raw files or disk)
Ext4 = ~ 16 TB (using 4 KB block size)
XFS = ~8 Exabytes qcow2 and host file systems keep their own metadata and scalability should be evaluated/tuned when trying very large image sizes. Using raw disks means fewer layers that could affect scalability or max size.
libvirt uses a directory-based storage pool, the /var/l i b/l i bvi rt/i mag es/ directory, as the default storage pool. The default storage pool can be changed to another storage pool.
Lo cal st o rag e p o o ls - Local storage pools are directly attached to the host physical machine server. Local storage pools include: local directories, directly attached disks, physical partitions, and LVM volume groups. These storage volumes store guest virtual machine images or are attached to guest virtual machines as additional storage. As local storage pools are directly attached to the host physical machine server, they are useful for development, testing and small deployments that do not require migration or large numbers of guest virtual machines. Local storage pools are not suitable for many production environments as local storage pools do not support live migration.
Net wo rked ( sh ared ) st o rag e p o o ls - Networked storage pools include storage devices shared over a network using standard protocols. Networked storage is required when migrating virtual machines between host physical machines with virt-manager, but is optional when migrating with virsh. Networked storage pools are managed by libvirt. Supported protocols for networked storage pools include:
Fibre Channel-based LUNs iSCSI
NFS
GFS2
SCSI RDMA protocols (SCSI RCP), the block export protocol used in InfiniBand and 10GbE iWARP adapters.
105
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
Multi-path storage pools should not be created or used as they are not fully supported.
12.2. Volumes
Storage pools are divided into storage volumes. Storage volumes are an abstraction of physical partitions, LVM logical volumes, file-based disk images and other storage types handled by libvirt.
Storage volumes are presented to guest virtual machines as local storage devices regardless of the underlying hardware.
Ref eren cin g vo lu mes
To reference a specific volume, three approaches are possible:
T h e n ame o f t h e vo lu me an d t h e st o rag e p o o l
A volume may be referred to by name, along with an identifier for the storage pool it belongs in. On the virsh command line, this takes the form --pool storage_pool volume_name.
For example, a volume named firstimage in the guest_images pool.
# virsh vol-info --pool guest_images firstimage
Name: firstimage
Type: block
Capacity: 20.00 GB
Allocation: 20.00 GB virsh #
T h e f u ll p at h t o t h e st o rag e o n t h e h o st p h ysical mach in e syst em
A volume may also be referred to by its full path on the file system. When using this approach, a pool identifier does not need to be included.
For example, a volume named secondimage.img, visible to the host physical machine system as /images/secondimage.img. The image can be referred to as /images/secondimage.img.
# virsh vol-info /images/secondimage.img
Name: secondimage.img
Type: file
Capacity: 20.00 GB
Allocation: 136.00 kB
T h e u n iq u e vo lu me key
When a volume is first created in the virtualization system, a unique identifier is generated and assigned to it. The unique identifier is termed the volume key. The format of this volume key varies upon the storage used.
When used with block based storage such as LVM, the volume key may follow this format: c3pKz4-qPVc-Xf7M-7WNM-WJc8-qSiz-mtvpGn
106
Chapt er 1 2 . St orage concept s
When used with file based storage, the volume key may instead be a copy of the full path to the volume storage.
/images/secondimage.img
For example, a volume with the volume key of Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr:
# virsh vol-info Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr
Name: firstimage
Type: block
Capacity: 20.00 GB
Allocation: 20.00 GB
vi rsh provides commands for converting between a volume name, volume path, or volume key:
vo l- n ame
Returns the volume name when provided with a volume path or volume key.
# virsh vol-name /dev/guest_images/firstimage
firstimage
# virsh vol-name Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr
vo l- p at h
Returns the volume path when provided with a volume key, or a storage pool identifier and volume name.
# virsh vol-path Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr
/dev/guest_images/firstimage
# virsh vol-path --pool guest_images firstimage
/dev/guest_images/firstimage
T h e vo l- key co mman d
Returns the volume key when provided with a volume path, or a storage pool identifier and volume name.
# virsh vol-key /dev/guest_images/firstimage
Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr
# virsh vol-key --pool guest_images firstimage
Wlvnf7-a4a3-Tlje-lJDa-9eak-PZBv-LoZuUr
107
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 13. Storage pools
This chapter includes instructions on creating storage pools of assorted types. A storage pool is a quantity of storage set aside by an administrator, often a dedicated storage administrator, for use by virtual machines. Storage pools are often divided into storage volumes either by the storage administrator or the system administrator, and the volumes are assigned to guest virtual machines as block devices.
Examp le 13.1. NFS st o rag e p o o l
Suppose a storage administrator responsible for an NFS server creates a share to store guest virtual machines' data. The system administrator defines a pool on the host physical machine with the details of the share (nfs.example.com:/path/to /share should be mounted on /vm_d ata).
When the pool is started, libvirt mounts the share on the specified directory, just as if the system administrator logged in and executed mo unt nfs. exampl e. co m: /path/to /share /vmd ata.
If the pool is configured to autostart, libvirt ensures that the NFS share is mounted on the directory specified when libvirt is started.
Once the pool starts, the files that the NFS share, are reported as volumes, and the storage volumes' paths are then queried using the libvirt APIs. The volumes' paths can then be copied into the section of a guest virtual machine's XML definition file describing the source storage for the guest virtual machine's block devices. With NFS, applications using the libvirt APIs can create and delete volumes in the pool (files within the NFS share) up to the limit of the size of the pool (the maximum storage capacity of the share). Not all pool types support creating and deleting volumes.
Stopping the pool negates the start operation, in this case, unmounts the NFS share. The data on the share is not modified by the destroy operation, despite the name. See man virsh for more details.
Note
Storage pools and volumes are not required for the proper operation of guest virtual machines. Pools and volumes provide a way for libvirt to ensure that a particular piece of storage will be available for a guest virtual machine, but some administrators will prefer to manage their own storage and guest virtual machines will operate properly without any pools or volumes defined. On systems that do not use pools, system administrators must ensure the availability of the guest virtual machines' storage using whatever tools they prefer, for example, adding the NFS share to the host physical machine's fstab so that the share is mounted at boot time.
13.1. Disk-based st orage pools
This section covers creating disk based storage devices for guest virtual machines.
108
Chapt er 1 3. St orage pools
Warning
Guests should not be given write access to whole disks or block devices (for example,
/d ev/sd b). Use partitions (for example, /d ev/sd b1) or LVM volumes.
If you pass an entire block device to the guest, the guest will likely partition it or create its own
LVM groups on it. This can cause the host physical machine physical machine to detect these partitions or LVM groups and cause errors.
13.1.1. Creat ing a disk based st orage pool using virsh
This procedure creates a new storage pool using a disk device with the vi rsh command.
Warning
Dedicating a disk to a storage pool will reformat and erase all data presently stored on the disk device! It is strongly recommended to back up the storage device before commencing with the following procedure:
1. Creat e a G PT d isk lab el o n t h e d isk
The disk must be relabeled with a GUID Partition Table (GPT) disk label. GPT disk labels allow for creating a large numbers of partitions, up to 128 partitions, on each device. GPT partition tables can store partition data for far more partitions than the MS-DOS partition table.
# parted /dev/sdb
GNU Parted 2.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel
New disk label type? gpt
(parted) quit
Information: You may need to update /etc/fstab.
#
2. Creat e t h e st o rag e p o o l co n f ig u rat io n f ile
Create a temporary XML text file containing the storage pool information required for the new device.
The file must be in the format shown below, and contain the following fields:
<n ame>g u est _imag es_d isk</n ame>
The name parameter determines the name of the storage pool. This example uses the name guest_images_disk in the example below.
<d evice p at h = ' /dev/sdb' />
The device parameter with the path attribute specifies the device path of the storage device. This example uses the device /dev/sdb.
109
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<t arg et > <p at h >/dev</p at h ></t arg et >
The file system target parameter with the path sub-parameter determines the location on the host physical machine file system to attach volumes created with this storage pool.
For example, sdb1, sdb2, sdb3. Using /dev/, as in the example below, means volumes created from this storage pool can be accessed as /dev/sdb1, /dev/sdb2,
/dev/sdb3.
<f o rmat t yp e= ' gpt' />
The format parameter specifies the partition table type. This example uses the gpt in the example below, to match the GPT disk label type created in the previous step.
Create the XML file for the storage pool device with a text editor.
Examp le 13.2. Disk b ased st o rag e d evice st o rag e p o o l
<pool type='disk'>
<name>guest_images_disk</name>
<source>
<device path='/dev/sdb'/>
<format type='gpt'/>
</source>
<target>
<path>/dev</path>
</target>
</pool>
3. At t ach t h e d evice
Add the storage pool definition using the vi rsh po o l -d efi ne command with the XML configuration file created in the previous step.
# virsh pool-define ~/guest_images_disk.xml
Pool guest_images_disk defined from /root/guest_images_disk.xml
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images_disk inactive no
4. St art t h e st o rag e p o o l
Start the storage pool with the vi rsh po o l -start command. Verify the pool is started with the vi rsh po o l -l i st --al l command.
# virsh pool-start guest_images_disk
Pool guest_images_disk started
# virsh pool-list --all
Name State Autostart
110
Chapt er 1 3. St orage pools
----------------------------------------default active yes guest_images_disk active no
5. T u rn o n au t o st art
Turn on autostart for the storage pool. Autostart configures the l i bvi rtd service to start the storage pool when the service starts.
# virsh pool-autostart guest_images_disk
Pool guest_images_disk marked as autostarted
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images_disk active yes
6. Verif y t h e st o rag e p o o l co n f ig u rat io n
Verify the storage pool was created correctly, the sizes reported correctly, and the state reports as runni ng .
# virsh pool-info guest_images_disk
Name: guest_images_disk
UUID: 551a67c8-5f2a-012c-3844-df29b167431c
State: running
Capacity: 465.76 GB
Allocation: 0.00
Available: 465.76 GB
# ls -la /dev/sdb brw-rw----. 1 root disk 8, 16 May 30 14:08 /dev/sdb
# virsh vol-list guest_images_disk
Name Path
-----------------------------------------
7. O p t io n al: Remo ve t h e t emp o rary co n f ig u rat io n f ile
Remove the temporary storage pool XML configuration file if it is not needed.
# rm ~/guest_images_disk.xml
A disk based storage pool is now available.
13.1.2. Delet ing a st orage pool using virsh
The following demonstrates how to delete a storage pool using virsh:
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it.
# virsh pool-destroy guest_images_disk
2. Remove the storage pool's definition
111
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# virsh pool-undefine guest_images_disk
13.2. Part it ion-based st orage pools
This section covers using a pre-formatted block device, a partition, as a storage pool.
For the following examples, a host physical machine has a 500GB hard drive (/d ev/sd c) partitioned into one 500GB, ext4 formatted partition (/d ev/sd c1). We set up a storage pool for it using the procedure below.
13.2.1. Creat ing a part it ion-based st orage pool using virt -manager
This procedure creates a new storage pool using a partition of a storage device.
Pro ced u re 13.1. Creat in g a p art it io n - b ased st o rag e p o o l wit h virt - man ag er
1. O p en t h e st o rag e p o o l set t in g s a. In the vi rt-manag er graphical interface, select the host physical machine from the main window.
Open the Ed i t menu and select C o nnecti o n D etai l s
Fig u re 13.1. Co n n ect io n Det ails
b. Click on the Sto rag e tab of the C o nnecti o n D etai l s window.
112
Chapt er 1 3. St orage pools
Fig u re 13.2. St o rag e t ab
2. Creat e t h e n ew st o rag e p o o l a. Ad d a n ew p o o l ( p art 1)
Press the + button (the add pool button). The Ad d a New Sto rag e P o o l wizard appears.
Choose a Name for the storage pool. This example uses the name guest_images_fs.
Change the T ype to fs: P re-Fo rmatted Bl o ck D evi ce.
113
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.3. St o rag e p o o l n ame an d t yp e
Press the Fo rward button to continue.
b. Ad d a n ew p o o l ( p art 2)
Change the T arg et P ath, Fo rmat, and So urce P ath fields.
114
Fig u re 13.4 . St o rag e p o o l p at h an d f o rmat
T arg et Pat h
Enter the location to mount the source device for the storage pool in the
T arg et P ath field. If the location does not already exist, vi rt-manag er will create the directory.
Fo rmat
Select a format from the Fo rmat list. The device is formatted with the selected format.
This example uses the ext4 file system, the default Red Hat Enterprise Linux file system.
So u rce Pat h
Enter the device in the So urce P ath field.
This example uses the /dev/sdc1 device.
Verify the details and press the Fi ni sh button to create the storage pool.
3. Verif y t h e n ew st o rag e p o o l
Chapt er 1 3. St orage pools
The new storage pool appears in the storage list on the left after a few seconds. Verify the size is reported as expected, 458.20 GB Free in this example. Verify the State field reports the new storage pool as Active.
Select the storage pool. In the Auto start field, click the O n Bo o t checkbox. This will make sure the storage device starts whenever the l i bvi rtd service starts.
Fig u re 13.5. St o rag e list co n f irmat io n
The storage pool is now created, close the C o nnecti o n D etai l s window.
13.2.2. Delet ing a st orage pool using virt -manager
This procedure demonstrates how to delete a storage pool.
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it. To do this, select the storage pool you want to stop and click the red X icon at the bottom of the Storage window.
115
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.6 . St o p Ico n
2. Delete the storage pool by clicking the Trash can icon. This icon is only enabled if you stop the storage pool first.
13.2.3. Creat ing a part it ion-based st orage pool using virsh
This section covers creating a partition-based storage pool with the vi rsh command.
Warning
Do not use this procedure to assign an entire disk as a storage pool (for example,
/d ev/sd b). Guests should not be given write access to whole disks or block devices. Only use this method to assign partitions (for example, /d ev/sd b1) to storage pools.
Pro ced u re 13.2. Creat in g p re- f o rmat t ed b lo ck d evice st o rag e p o o ls u sin g virsh
1. Creat e t h e st o rag e p o o l d ef in it io n
Use the virsh po o l -d efi ne-as command to create a new storage pool definition. There are three options that must be provided to define a pre-formatted disk as a storage pool:
Part it io n n ame
The name parameter determines the name of the storage pool. This example uses the name guest_images_fs in the example below.
d evice
116
Chapt er 1 3. St orage pools
The device parameter with the path attribute specifies the device path of the storage device. This example uses the partition /dev/sdc1.
mo u n t p o in t
The mountpoint on the local file system where the formatted device will be mounted. If the mount point directory does not exist, the vi rsh command can create the directory.
The directory /guest_images is used in this example.
# virsh pool-define-as guest_images_fs fs - - /dev/sdc1 -
"/guest_images"
Pool guest_images_fs defined
The new pool and mount points are now created.
2. Verif y t h e n ew p o o l
List the present storage pools.
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes
guest_images_fs inactive no
3. Creat e t h e mo u n t p o in t
Use the vi rsh po o l -bui l d command to create a mount point for a pre-formatted file system storage pool.
# virsh pool-build guest_images_fs
Pool guest_images_fs built
# ls -la /guest_images total 8 drwx------. 2 root root 4096 May 31 19:38 .
dr-xr-xr-x. 25 root root 4096 May 31 19:38 ..
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images_fs inactive no
4. St art t h e st o rag e p o o l
Use the vi rsh po o l -start command to mount the file system onto the mount point and make the pool available for use.
# virsh pool-start guest_images_fs
Pool guest_images_fs started
# virsh pool-list --all
Name State Autostart
117
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
----------------------------------------default active yes guest_images_fs active no
5. T u rn o n au t o st art
By default, a storage pool defined with vi rsh, is not set to automatically start each time
l i bvi rtd starts. To remedy this, enable the automatic start with the vi rsh po o l -
auto start command. The storage pool is now automatically started each time l i bvi rtd starts.
# virsh pool-autostart guest_images_fs
Pool guest_images_fs marked as autostarted
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images_fs active yes
6. Verif y t h e st o rag e p o o l
Verify the storage pool was created correctly, the sizes reported are as expected, and the state is reported as runni ng . Verify there is a "lost+found" directory in the mount point on the file system, indicating the device is mounted.
# virsh pool-info guest_images_fs
Name: guest_images_fs
UUID: c7466869-e82a-a66c-2187-dc9d6f0877d0
State: running
Persistent: yes
Autostart: yes
Capacity: 458.39 GB
Allocation: 197.91 MB
Available: 458.20 GB
# mount | grep /guest_images
/dev/sdc1 on /guest_images type ext4 (rw)
# ls -la /guest_images total 24 drwxr-xr-x. 3 root root 4096 May 31 19:47 .
dr-xr-xr-x. 25 root root 4096 May 31 19:38 ..
drwx------. 2 root root 16384 May 31 14:18 lost+found
13.2.4 . Delet ing a st orage pool using virsh
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it.
# virsh pool-destroy guest_images_disk
2. Optionally, if you want to remove the directory where the storage pool resides use the following command:
# virsh pool-delete guest_images_disk
118
Chapt er 1 3. St orage pools
3. Remove the storage pool's definition
# virsh pool-undefine guest_images_disk
13.3. Direct ory-based st orage pools
This section covers storing guest virtual machines in a directory on the host physical machine.
Directory-based storage pools can be created with vi rt-manag er or the vi rsh command line tools.
13.3.1. Creat ing a direct ory-based st orage pool wit h virt -manager
1. Creat e t h e lo cal d irect o ry a. O p t io n al: Creat e a n ew d irect o ry f o r t h e st o rag e p o o l
Create the directory on the host physical machine for the storage pool. This example uses a directory named /guest virtual machine_images.
# mkdir /guest_images b. Set d irect o ry o wn ersh ip
Change the user and group ownership of the directory. The directory must be owned by the root user.
# chown root:root /guest_images c. Set d irect o ry p ermissio n s
Change the file permissions of the directory.
# chmod 700 /guest_images d. Verif y t h e ch an g es
Verify the permissions were modified. The output shows a correctly configured empty directory.
# ls -la /guest_images total 8 drwx------. 2 root root 4096 May 28 13:57 .
dr-xr-xr-x. 26 root root 4096 May 28 13:57 ..
2. Co n f ig u re SELin u x f ile co n t ext s
Configure the correct SELinux context for the new directory. Note that the name of the pool and the directory do not have to match. However, when you shutdown the guest virtual machine, libvirt has to set the context back to a default value. The context of the directory determines what this default value is. It is worth explicitly labeling the directory virt_image_t,
119
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
so that when the guest virtual machine is shutdown, the images get labeled 'virt_image_t' and are thus isolated from other processes running on the host physical machine.
# semanage fcontext -a -t virt_image_t '/guest_images(/.*)?'
# restorecon -R /guest_images
3. O p en t h e st o rag e p o o l set t in g s a. In the vi rt-manag er graphical interface, select the host physical machine from the main window.
Open the Ed i t menu and select C o nnecti o n D etai l s
Fig u re 13.7. Co n n ect io n d et ails win d o w
b. Click on the Sto rag e tab of the C o nnecti o n D etai l s window.
120
Chapt er 1 3. St orage pools
Fig u re 13.8. St o rag e t ab
4. Creat e t h e n ew st o rag e p o o l a. Ad d a n ew p o o l ( p art 1)
Press the + button (the add pool button). The Ad d a New Sto rag e P o o l wizard appears.
Choose a Name for the storage pool. This example uses the name guest_images.
Change the T ype to d i r: Fi l esystem D i recto ry.
Fig u re 13.9 . Name t h e st o rag e p o o l
121
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Press the Fo rward button to continue.
b. Ad d a n ew p o o l ( p art 2)
Change the T arg et P ath field. For example, /guest_images.
Verify the details and press the Fi ni sh button to create the storage pool.
5. Verif y t h e n ew st o rag e p o o l
The new storage pool appears in the storage list on the left after a few seconds. Verify the size is reported as expected, 36.41 GB Free in this example. Verify the State field reports the new storage pool as Active.
Select the storage pool. In the Auto start field, confirm that the O n Bo o t checkbox is checked. This will make sure the storage pool starts whenever the l i bvi rtd service starts.
Fig u re 13.10. Verif y t h e st o rag e p o o l in f o rmat io n
The storage pool is now created, close the C o nnecti o n D etai l s window.
13.3.2. Delet ing a st orage pool using virt -manager
This procedure demonstrates how to delete a storage pool.
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it. To do this, select the storage pool you want to stop and click the red X icon at the bottom of the Storage window.
122
Chapt er 1 3. St orage pools
Fig u re 13.11. St o p Ico n
2. Delete the storage pool by clicking the Trash can icon. This icon is only enabled if you stop the storage pool first.
13.3.3. Creat ing a direct ory-based st orage pool wit h virsh
1. Creat e t h e st o rag e p o o l d ef in it io n
Use the vi rsh po o l -d efi ne-as command to define a new storage pool. There are two options required for creating directory-based storage pools:
The name of the storage pool.
This example uses the name guest_images. All further vi rsh commands used in this example use this name.
The path to a file system directory for storing guest image files. If this directory does not exist, vi rsh will create it.
This example uses the /guest_images directory.
# virsh pool-define-as guest_images dir - - - - "/guest_images"
Pool guest_images defined
2. Verif y t h e st o rag e p o o l is list ed
Verify the storage pool object is created correctly and the state reports it as i nacti ve.
# virsh pool-list --all
Name State Autostart
123
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
----------------------------------------default active yes guest_images inactive no
3. Creat e t h e lo cal d irect o ry
Use the vi rsh po o l -bui l d command to build the directory-based storage pool for the directory guest_images (for example), as shown:
# virsh pool-build guest_images
Pool guest_images built
# ls -la /guest_images total 8 drwx------. 2 root root 4096 May 30 02:44 .
dr-xr-xr-x. 26 root root 4096 May 30 02:44 ..
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images inactive no
4. St art t h e st o rag e p o o l
Use the virsh command po o l -start to enable a directory storage pool, thereby allowing allowing volumes of the pool to be used as guest disk images.
# virsh pool-start guest_images
Pool guest_images started
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images active no
5. T u rn o n au t o st art
Turn on autostart for the storage pool. Autostart configures the l i bvi rtd service to start the storage pool when the service starts.
# virsh pool-autostart guest_images
Pool guest_images marked as autostarted
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images active yes
6. Verif y t h e st o rag e p o o l co n f ig u rat io n
Verify the storage pool was created correctly, the size is reported correctly, and the state is reported as runni ng . If you want the pool to be accessible even if the guest virtual machine is not running, make sure that P ersi stent is reported as yes. If you want the pool to start automatically when the service starts, make sure that Auto start is reported as yes.
124
Chapt er 1 3. St orage pools
# virsh pool-info guest_images
Name: guest_images
UUID: 779081bf-7a82-107b-2874-a19a9c51d24c
State: running
Persistent: yes
Autostart: yes
Capacity: 49.22 GB
Allocation: 12.80 GB
Available: 36.41 GB
# ls -la /guest_images total 8 drwx------. 2 root root 4096 May 30 02:44 .
dr-xr-xr-x. 26 root root 4096 May 30 02:44 ..
#
A directory-based storage pool is now available.
13.3.4 . Delet ing a st orage pool using virsh
The following demonstrates how to delete a storage pool using virsh:
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it.
# virsh pool-destroy guest_images_disk
2. Optionally, if you want to remove the directory where the storage pool resides use the following command:
# virsh pool-delete guest_images_disk
3. Remove the storage pool's definition
# virsh pool-undefine guest_images_disk
13.4 . LVM-based st orage pools
This chapter covers using LVM volume groups as storage pools.
LVM-based storage groups provide the full flexibility of LVM.
Note
Thin provisioning is currently not possible with LVM based storage pools.
125
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
Please refer to the Red Hat Enterprise Linux Storage Administration Guide for more details on LVM.
Warning
LVM-based storage pools require a full disk partition. If activating a new partition/device with these procedures, the partition will be formatted and all data will be erased. If using the host's existing Volume Group (VG) nothing will be erased. It is recommended to back up the storage device before commencing the following procedure.
13.4 .1. Creat ing an LVM-based st orage pool wit h virt -manager
LVM-based storage pools can use existing LVM volume groups or create new LVM volume groups on a blank partition.
1. O p t io n al: Creat e n ew p art it io n f o r LVM vo lu mes
These steps describe how to create a new partition and LVM volume group on a new hard disk drive.
Warning
This procedure will remove all data from the selected storage device.
a. Creat e a n ew p art it io n
Use the fd i sk command to create a new disk partition from the command line. The following example creates a new partition that uses the entire disk on the storage device /d ev/sd b.
# fdisk /dev/sdb
Command (m for help):
Press n for a new partition.
b. Press p for a primary partition.
Command action
e extended
p primary partition (1-4) c. Choose an available partition number. In this example the first partition is chosen by entering 1.
Partition number (1-4): 1 d. Enter the default first cylinder by pressing Enter.
126
Chapt er 1 3. St orage pools
First cylinder (1-400, default 1): e. Select the size of the partition. In this example the entire disk is allocated by pressing
Enter.
Last cylinder or +size or +sizeM or +sizeK (2-400, default
400): f. Set the type of partition by pressing t.
Command (m for help): t g. Choose the partition you created in the previous steps. In this example, the partition number is 1.
Partition number (1-4): 1 h. Enter 8e for a Linux LVM partition.
Hex code (type L to list codes): 8e i. write changes to disk and quit.
Command (m for help): w
Command (m for help): q j. Creat e a n ew LVM vo lu me g ro u p
Create a new LVM volume group with the vg create command. This example creates a volume group named guest_images_lvm.
# vgcreate guest_images_lvm /dev/sdb1
Physical volume "/dev/vdb1" successfully created
Volume group "guest_images_lvm" successfully created
The new LVM volume group, guest_images_lvm, can now be used for an LVM-based storage pool.
2. O p en t h e st o rag e p o o l set t in g s a. In the vi rt-manag er graphical interface, select the host from the main window.
Open the Ed i t menu and select C o nnecti o n D etai l s
127
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.12. Co n n ect io n d et ails
b. Click on the Sto rag e tab.
128
Fig u re 13.13. St o rag e t ab
3. Creat e t h e n ew st o rag e p o o l a. St art t h e Wiz ard
Press the + button (the add pool button). The Ad d a New Sto rag e P o o l wizard appears.
Choose a Name for the storage pool. We use guest_images_lvm for this example. Then change the T ype to l o g i cal : LVM Vo l ume G ro up, and
Chapt er 1 3. St orage pools
Fig u re 13.14 . Ad d LVM st o rag e p o o l
Press the Fo rward button to continue.
b. Ad d a n ew p o o l ( p art 2)
Change the T arg et P ath field. This example uses /guest_images.
Now fill in the T arg et P ath and So urce P ath fields, then tick the Bui l d P o o l check box.
Use the T arg et P ath field to either select an existing LVM volume group or as the name for a new volume group. The default format is /d ev/storage_pool_name.
This example uses a new volume group named /dev/guest_images_lvm.
The So urce P ath field is optional if an existing LVM volume group is used in the
T arg et P ath.
For new LVM volume groups, input the location of a storage device in the So urce
P ath field. This example uses a blank partition /dev/sdc.
The Bui l d P o o l checkbox instructs vi rt-manag er to create a new LVM volume group. If you are using an existing volume group you should not select the
Bui l d P o o l checkbox.
This example is using a blank partition to create a new volume group so the
Bui l d P o o l checkbox must be selected.
129
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.15. Ad d t arg et an d so u rce
Verify the details and press the Fi ni sh button format the LVM volume group and create the storage pool.
c. Co n f irm t h e d evice t o b e f o rmat t ed
A warning message appears.
130
Fig u re 13.16 . Warn in g messag e
Press the Y es button to proceed to erase all data on the storage device and create the storage pool.
4. Verif y t h e n ew st o rag e p o o l
The new storage pool will appear in the list on the left after a few seconds. Verify the details are what you expect, 465.76 GB Free in our example. Also verify the State field reports the new storage pool as Active.
It is generally a good idea to have the Auto start check box enabled, to ensure the storage pool starts automatically with libvirtd.
Chapt er 1 3. St orage pools
Fig u re 13.17. Co n f irm LVM st o rag e p o o l d et ails
Close the Host Details dialog, as the task is now complete.
13.4 .2. Delet ing a st orage pool using virt -manager
This procedure demonstrates how to delete a storage pool.
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it. To do this, select the storage pool you want to stop and click the red X icon at the bottom of the Storage window.
131
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.18. St o p Ico n
2. Delete the storage pool by clicking the Trash can icon. This icon is only enabled if you stop the storage pool first.
13.4 .3. Creat ing an LVM-based st orage pool wit h virsh
This section outlines the steps required to create an LVM-based storage pool with the vi rsh command. It uses the example of a pool named g u est _imag es_lvm from a single drive
(/d ev/sd c). This is only an example and your settings should be substituted as appropriate.
Pro ced u re 13.3. Creat in g an LVM- b ased st o rag e p o o l wit h virsh
1. Define the pool name g u est _imag es_lvm.
# virsh pool-define-as guest_images_lvm logical - - /dev/sdc
libvirt_lvm \ /dev/libvirt_lvm
Pool guest_images_lvm defined
2. Build the pool according to the specified name. If you are using an already existing volume group, skip this step.
# virsh pool-build guest_images_lvm
Pool guest_images_lvm built
3. Initialize the new pool.
132
Chapt er 1 3. St orage pools
# virsh pool-start guest_images_lvm
Pool guest_images_lvm started
4. Show the volume group information with the vg s command.
# vgs
VG #PV #LV #SN Attr VSize VFree libvirt_lvm 1 0 0 wz--n- 465.76g 465.76g
5. Set the pool to start automatically.
# virsh pool-autostart guest_images_lvm
Pool guest_images_lvm marked as autostarted
6. List the available pools with the vi rsh command.
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes guest_images_lvm active yes
7. The following commands demonstrate the creation of three volumes (volume1, volume2 and
volume3) within this pool.
# virsh vol-create-as guest_images_lvm volume1 8G
Vol volume1 created
# virsh vol-create-as guest_images_lvm volume2 8G
Vol volume2 created
# virsh vol-create-as guest_images_lvm volume3 8G
Vol volume3 created
8. List the available volumes in this pool with the vi rsh command.
# virsh vol-list guest_images_lvm
Name Path
----------------------------------------volume1 /dev/libvirt_lvm/volume1 volume2 /dev/libvirt_lvm/volume2 volume3 /dev/libvirt_lvm/volume3
9. The following two commands (l vscan and l vs) display further information about the newly created volumes.
# lvscan
ACTIVE '/dev/libvirt_lvm/volume1' [8.00 GiB] inherit
ACTIVE '/dev/libvirt_lvm/volume2' [8.00 GiB] inherit
ACTIVE '/dev/libvirt_lvm/volume3' [8.00 GiB] inherit
# lvs
133
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
LV VG Attr LSize Pool Origin Data% Move Log
Copy% Convert volume1 libvirt_lvm -wi-a- 8.00g
volume2 libvirt_lvm -wi-a- 8.00g
volume3 libvirt_lvm -wi-a- 8.00g
13.4 .4 . Delet ing a st orage pool using virsh
The following demonstrates how to delete a storage pool using virsh:
1. To avoid any issues with other guests using the same pool, it is best to stop the storage pool and release any resources in use by it.
# virsh pool-destroy guest_images_disk
2. Optionally, if you want to remove the directory where the storage pool resides use the following command:
# virsh pool-delete guest_images_disk
3. Remove the storage pool's definition
# virsh pool-undefine guest_images_disk
13.5. iSCSI-based st orage pools
This section covers using iSCSI-based devices to store guest virtual machines.
iSCSI (Internet Small Computer System Interface) is a network protocol for sharing storage devices.
iSCSI connects initiators (storage clients) to targets (storage servers) using SCSI instructions over the IP layer.
13.5.1. Configuring a soft ware iSCSI t arget
The scsi-target-utils package provides a tool for creating software-backed iSCSI targets.
Pro ced u re 13.4 . Creat in g an iSCSI t arg et
1. In st all t h e req u ired p ackag es
Install the scsi-target-utils package and all dependencies
# yum install scsi-target-utils
2. St art t h e t g t d service
The tg td service host physical machines SCSI targets and uses the iSCSI protocol to host physical machine targets. Start the tg td service and make the service persistent after restarting with the chkco nfi g command.
# service tgtd start
# chkconfig tgtd on
134
Chapt er 1 3. St orage pools
3. O p t io n al: Creat e LVM vo lu mes
LVM volumes are useful for iSCSI backing images. LVM snapshots and resizing can be beneficial for guest virtual machines. This example creates an LVM image named virtimage1 on a new volume group named virtstore on a RAID5 array for hosting guest virtual machines with iSCSI.
a. Creat e t h e RAID array
Creating software RAID5 arrays is covered by the Red Hat Enterprise Linux Deployment
Guide.
b. Creat e t h e LVM vo lu me g ro u p
Create a volume group named virtstore with the vg create command.
# vgcreate virtstore /dev/md1 c. Creat e a LVM lo g ical vo lu me
Create a logical volume group named virtimage1 on the virtstore volume group with a size of 20GB using the l vcreate command.
# lvcreate --size 20G -n virtimage1 virtstore
The new logical volume, virtimage1, is ready to use for iSCSI.
4. O p t io n al: Creat e f ile- b ased imag es
File-based storage is sufficient for testing but is not recommended for production environments or any significant I/O activity. This optional procedure creates a file based imaged named virtimage2.img for an iSCSI target.
a. Creat e a n ew d irect o ry f o r t h e imag e
Create a new directory to store the image. The directory must have the correct SELinux contexts.
# mkdir -p /var/lib/tgtd/virtualization b. Creat e t h e imag e f ile
Create an image named virtimage2.img with a size of 10GB.
# dd if=/dev/zero of=/var/lib/tgtd/virtualization/virtimage2.img bs=1M seek=10000 count=0 c. Co n f ig u re SELin u x f ile co n t ext s
Configure the correct SELinux context for the new image and directory.
# restorecon -R /var/lib/tgtd
135
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The new file-based image, virtimage2.img, is ready to use for iSCSI.
5.
Creat e t arg et s
Targets can be created by adding a XML entry to the /etc/tg t/targ ets. co nf file. The
targ et attribute requires an iSCSI Qualified Name (IQN). The IQN is in the format: iqn.yyyy-mm.reversed domain name:optional identifier text
Where:
yyyy-mm represents the year and month the device was started (for example: 2010-05);
reversed domain name is the host physical machines domain name in reverse (for example
server1.example.com in an IQN would be com.example.server1); and
optional identifier text is any text string, without spaces, that assists the administrator in identifying devices or hardware.
This example creates iSCSI targets for the two types of images created in the optional steps on server1.example.com with an optional identifier trial. Add the following to the
/etc/tg t/targ ets. co nf file.
<target iqn.2010-05.com.example.server1:iscsirhel6guest>
backing-store /dev/virtstore/virtimage1 #LUN 1
backing-store /var/lib/tgtd/virtualization/virtimage2.img #LUN
2
write-cache off
</target>
Ensure that the /etc/tg t/targ ets. co nf file contains the d efaul t-d ri ver i scsi line to set the driver type as iSCSI. The driver uses iSCSI by default.
Important
This example creates a globally accessible target without access control. Refer to the scsi-target-utils for information on implementing secure access.
6. Rest art t h e t g t d service
Restart the tg td service to reload the configuration changes.
# service tgtd restart
7. ip t ab les co n f ig u rat io n
Open port 3260 for iSCSI access with i ptabl es.
# iptables -I INPUT -p tcp -m tcp --dport 3260 -j ACCEPT
# service iptables save
# service iptables restart
136
Chapt er 1 3. St orage pools
8. Verif y t h e n ew t arg et s
View the new targets to ensure the setup was successful with the tg t-ad mi n --sho w command.
# tgt-admin --show
Target 1: iqn.2010-05.com.example.server1:iscsirhel6guest
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 20000 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dev/virtstore/virtimage1
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 10000 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /var/lib/tgtd/virtualization/virtimage2.img
Account information:
ACL information:
ALL
Warning
The ACL list is set to all. This allows all systems on the local network to access this device. It is recommended to set host physical machine access ACLs for production environments.
9. O p t io n al: T est d isco very
Test whether the new iSCSI device is discoverable.
137
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# iscsiadm --mode discovery --type sendtargets --portal server1.example.com
127.0.0.1:3260,1 iqn.2010-05.com.example.server1:iscsirhel6guest
10. O p t io n al: T est at t ach in g t h e d evice
Attach the new device (iqn.2010-05.com.example.server1:iscsirhel6guest) to determine whether the device can be attached.
# iscsiadm -d2 -m node --login scsiadm: Max file limits 1024 1024
Logging in to [iface: default, target: iqn.2010-
05.com.example.server1:iscsirhel6guest, portal: 10.0.0.1,3260]
Login to [iface: default, target: iqn.2010-
05.com.example.server1:iscsirhel6guest, portal: 10.0.0.1,3260] successful.
Detach the device.
# iscsiadm -d2 -m node --logout scsiadm: Max file limits 1024 1024
Logging out of session [sid: 2, target: iqn.2010-
05.com.example.server1:iscsirhel6guest, portal: 10.0.0.1,3260
Logout of [sid: 2, target: iqn.2010-
05.com.example.server1:iscsirhel6guest, portal: 10.0.0.1,3260] successful.
An iSCSI device is now ready to use for virtualization.
13.5.2. Adding an iSCSI t arget t o virt -manager
This procedure covers creating a storage pool with an iSCSI target in vi rt-manag er.
Pro ced u re 13.5. Ad d in g an iSCSI d evice t o virt - man ag er
1. O p en t h e h o st p h ysical mach in e' s st o rag e t ab
Open the Sto rag e tab in the Ho st D etai l s window.
a. Open vi rt-manag er.
b. Select a host physical machine from the main vi rt-manag er window. Click Ed i t
menu and select C o nnecti o n D etai l s.
138
Fig u re 13.19 . Co n n ect io n d et ails
c. Click on the Sto rag e tab.
Chapt er 1 3. St orage pools
Fig u re 13.20. St o rag e men u
2. Ad d a n ew p o o l ( p art 1)
Press the + button (the add pool button). The Ad d a New Sto rag e P o o l wizard appears.
139
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
14 0
Fig u re 13.21. Ad d an iscsi st o rag e p o o l n ame an d t yp e
Choose a name for the storage pool, change the Type to iscsi, and press Fo rward to continue.
3. Ad d a n ew p o o l ( p art 2)
You will need the information you used in Section 13.5, “ iSCSI-based storage pools”
and
to complete the fields in this menu.
a. Enter the iSCSI source and target. The Fo rmat option is not available as formatting is handled by the guest virtual machines. It is not advised to edit the T arg et P ath. The default target path value, /d ev/d i sk/by-path/, adds the drive path to that directory. The target path should be the same on all host physical machines for migration.
b. Enter the hostname or IP address of the iSCSI target. This example uses
ho st1. exampl e. co m.
c. In the So urce P athfield, enter the iSCSI target IQN. If you look at Step 5
in
Section 13.5, “ iSCSI-based storage pools”
, this is the information you added in the
/etc/tg t/targ ets. co nf fi l e. This example uses i q n. 20 10 -
0 5. co m. exampl e. server1: i scsi rhel 6 g uest.
d. Check the IQ N checkbox to enter the IQN for the initiator. This example uses
i q n. 20 10 -0 5. co m. exampl e. ho st1: i scsi rhel 6 .
e. Click Fi ni sh to create the new storage pool.
Chapt er 1 3. St orage pools
Fig u re 13.22. Creat e an iscsi st o rag e p o o l
13.5.3. Delet ing a st orage pool using virt -manager
This procedure demonstrates how to delete a storage pool.
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it. To do this, select the storage pool you want to stop and click the red X icon at the bottom of the Storage window.
14 1
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.23. St o p Ico n
2. Delete the storage pool by clicking the Trash can icon. This icon is only enabled if you stop the storage pool first.
13.5.4 . Creat ing an iSCSI-based st orage pool wit h virsh
1. Use p o o l- d ef in e- as t o d ef in e t h e p o o l f ro m t h e co mman d lin e
Storage pool definitions can be created with the vi rsh command line tool. Creating storage pools with vi rsh is useful for systems administrators using scripts to create multiple storage pools.
The vi rsh po o l -d efi ne-as command has several parameters which are accepted in the following format: virsh pool-define-as name type source-host source-path source-dev
source-name target
The parameters are explained as follows:
t yp e
defines this pool as a particular type, iscsi for example
n ame
must be unique and sets the name for the storage pool
so u rce- h o st an d so u rce- p at h
the hostname and iSCSI IQN respectively
14 2
Chapt er 1 3. St orage pools so u rce- d ev an d so u rce- n ame
these parameters are not required for iSCSI-based pools, use a - character to leave the field blank.
t arg et
defines the location for mounting the iSCSI device on the host physical machine
The example below creates the same iSCSI-based storage pool as the previous step.
# virsh pool-define-as --name scsirhel6guest --type iscsi \
--source-host server1.example.com \
--source-dev iqn.2010-05.com.example.server1:iscsirhel6guest
--target /dev/disk/by-path
Pool iscsirhel6guest defined
2. Verif y t h e st o rag e p o o l is list ed
Verify the storage pool object is created correctly and the state reports as i nacti ve.
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes
iscsirhel6guest inactive no
3. St art t h e st o rag e p o o l
Use the virsh command po o l -start for this. po o l -start enables a directory storage pool, allowing it to be used for volumes and guest virtual machines.
# virsh pool-start guest_images_disk
Pool guest_images_disk started
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes
iscsirhel6guest active no
4. T u rn o n au t o st art
Turn on autostart for the storage pool. Autostart configures the l i bvi rtd service to start the storage pool when the service starts.
# virsh pool-autostart iscsirhel6guest
Pool iscsirhel6guest marked as autostarted
Verify that the iscsirhel6guest pool has autostart set:
# virsh pool-list --all
Name State Autostart
----------------------------------------default active yes
iscsirhel6guest active yes
14 3
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
5. Verif y t h e st o rag e p o o l co n f ig u rat io n
Verify the storage pool was created correctly, the sizes reported correctly, and the state reports as runni ng .
# virsh pool-info iscsirhel6guest
Name: iscsirhel6guest
UUID: afcc5367-6770-e151-bcb3-847bc36c5e28
State: running
Persistent: unknown
Autostart: yes
Capacity: 100.31 GB
Allocation: 0.00
Available: 100.31 GB
An iSCSI-based storage pool is now available.
13.5.5. Delet ing a st orage pool using virsh
The following demonstrates how to delete a storage pool using virsh:
1. To avoid any issues with other guest virtual machines using the same pool, it is best to stop the storage pool and release any resources in use by it.
# virsh pool-destroy guest_images_disk
2. Remove the storage pool's definition
# virsh pool-undefine guest_images_disk
13.6. NFS-based st orage pools
This procedure covers creating a storage pool with a NFS mount point in vi rt-manag er.
13.6.1. Creat ing a NFS-based st orage pool wit h virt -manager
1. O p en t h e h o st p h ysical mach in e' s st o rag e t ab
Open the Sto rag e tab in the Ho st D etai l s window.
a. Open vi rt-manag er.
b. Select a host physical machine from the main vi rt-manag er window. Click Ed i t
menu and select C o nnecti o n D etai l s.
14 4
Fig u re 13.24 . Co n n ect io n d et ails
c. Click on the Storage tab.
Chapt er 1 3. St orage pools
Fig u re 13.25. St o rag e t ab
2. Creat e a n ew p o o l ( p art 1)
Press the + button (the add pool button). The Ad d a New Sto rag e P o o l wizard appears.
14 5
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
14 6
Fig u re 13.26 . Ad d an NFS n ame an d t yp e
Choose a name for the storage pool and press Fo rward to continue.
3. Creat e a n ew p o o l ( p art 2)
Enter the target path for the device, the hostname and the NFS share path. Set the Fo rmat option to NFS or auto (to detect the type). The target path must be identical on all host physical machines for migration.
Enter the hostname or IP address of the NFS server. This example uses
server1. exampl e. co m.
Enter the NFS path. This example uses /nfstri al .
Chapt er 1 3. St orage pools
Fig u re 13.27. Creat e an NFS st o rag e p o o l
Press Fi ni sh to create the new storage pool.
13.6.2. Delet ing a st orage pool using virt -manager
This procedure demonstrates how to delete a storage pool.
1. To avoid any issues with other guests using the same pool, it is best to stop the storage pool and release any resources in use by it. To do this, select the storage pool you want to stop and click the red X icon at the bottom of the Storage window.
14 7
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 13.28. St o p Ico n
2. Delete the storage pool by clicking the Trash can icon. This icon is only enabled if you stop the storage pool first.
13.7. Glust erFS st orage pools
This section covers enabling a GlusterFS based storage pool. Red Hat Enterprise Linux 6.5 includes native support for creating virtual machines with GlusterFS. GlusterFS is a userspace file system that uses FUSE. When enabled in a guest virtual machine it enables a KVM host physical machine to boot guest virtual machine images from one or more GlusterFS storage volumes, and to use images from a
GlusterFS storage volume as data disks for guest virtual machines.
Note
Refer to the Red Hat Storage Administration Guide for additonal information.
13.7.1. Creat ing a Glust erFS st orage pool using virsh
This section will demonstrate how to prepare a Gluster server and an active Gluster volume.
Pro ced u re 13.6 . Prep arin g a G lu st er server an d an act ive G lu st er vo lu me
1. Obtain the IP address of the Gluster server by listing its status with the following command:
# gluster volume status
Status of volume: gluster-vol1
14 8
Chapt er 1 3. St orage pools
Gluster process Port Online Pid
------------------------------------------------------------------
------------
Brick 222.111.222.111:/gluster-vol1 49155 Y 18634
Task Status of Volume gluster-vol1
------------------------------------------------------------------
------------
There are no active volume tasks
2. If you haven't already done so, install glusterfs-fuse and enable virt_use_fusefs. Then prepare one host which will connect to the Gluster server by running the following commands:
# setsebool virt_use_fusefs on
# getsebool virt_use_fusefs virt_use_fusefs --> on
3. Create a new XML file to configure a Gluster storage pool (named glusterfs-pool.xml in this example) specifying po o l type as gluster, and add the following data:
<pool type= 'gluster' >
<name> glusterfs-pool </name>
<source>
<host name= '111.222.111.222' />
<dir path= '/' />
<name> gluster-vol1 </name>
</source>
</pool>
Fig u re 13.29 . G lu st erFS XML f ile co n t en t s
4. Define and start the Gluster pool, using the following commands:
# virsh pool-define glusterfs-pool.xml
Pool gluster-pool defined from glusterfs-pool.xml
# virsh pool-list --all
Name State Autostart
----------------------------------------- gluster-pool inactive no
# virsh pool-start gluster-pool
Pool gluster-pool started
# virsh pool-list --all
Name State Autostart
----------------------------------------- gluster-pool active no
14 9
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# virsh vol-list gluster-pool
Name Path
----------------------------------------qcow2.img gluster://111.222.111.222/glustervol1/qcow2.img
raw.img gluster://111.222.111.222/gluster-vol1/raw.img
13.7.2. Delet ing a Glust erFS st orage pool using virsh
This section details how to delete a storage pool using virsh.
Pro ced u re 13.7. Delet in g a G lu st erFS st o rag e p o o l
1. Set the status of the storage pool to inactive, using the following command:
# virsh pool-destroy gluster-pool
Pool gluster-pool destroyed
2. Confirm the pool is inactive, using the following command
# virsh pool-list --all
Name State Autostart
----------------------------------------- gluster-pool inactive no
3. Undefine the GlusterFS storage pool using the following command:
# virsh pool-undefine gluster-pool
Pool gluster-pool has been undefined
4. Confirm the pool is undefined, using the following command:
# virsh pool-list --all
Name State Autostart
-----------------------------------------
150
Chapt er 1 4 . Volumes
Chapter 14. Volumes
14 .1. Creat ing volumes
This section shows how to create disk volumes inside a block based storage pool. In the example below, the vi rsh vo l -create-as command will create a storage volume with a specific size in GB within the guest_images_disk storage pool. As this command is repeated per volume needed, three volumes are created as shown in the example.
# virsh vol-create-as guest_images_disk volume1 8G
Vol volume1 created
# virsh vol-create-as guest_images_disk volume2 8G
Vol volume2 created
# virsh vol-create-as guest_images_disk volume3 8G
Vol volume3 created
# virsh vol-list guest_images_disk
Name Path
----------------------------------------volume1 /dev/sdb1 volume2 /dev/sdb2 volume3 /dev/sdb3
# parted -s /dev/sdb pri nt
Model: ATA ST3500418AS (scsi)
Disk /dev/sdb: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
2 17.4kB 8590MB 8590MB primary
3 8590MB 17.2GB 8590MB primary
1 21.5GB 30.1GB 8590MB primary
14 .2. Cloning volumes
The new volume will be allocated from storage in the same storage pool as the volume being cloned.
The vi rsh vo l -cl o ne must have the --po o l argument which dictates the name of the storage pool that contains the volume to be cloned. The rest of the command names the volume to be cloned
(volume3) and the name of the new volume that was cloned (clone1). The vi rsh vo l -l i st command lists the volumes that are present in the storage pool (guest_images_disk).
# virsh vol-clone --pool guest_images_disk volume3 clone1
Vol clone1 cloned from volume3
# vi rsh vo l -l i st guest_images_disk
Name Path
----------------------------------------volume1 /dev/sdb1 volume2 /dev/sdb2
151
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
volume3 /dev/sdb3 clone1 /dev/sdb4
# parted -s /dev/sdb pri nt
Model: ATA ST3500418AS (scsi)
Disk /dev/sdb: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size File system Name Flags
1 4211MB 12.8GB 8595MB primary
2 12.8GB 21.4GB 8595MB primary
3 21.4GB 30.0GB 8595MB primary
4 30.0GB 38.6GB 8595MB primary
14 .3. Adding st orage devices t o guest s
This section covers adding storage devices to a guest. Additional storage can only be added as needed.
14 .3.1. Adding file based st orage t o a guest
File-based storage is a collection of files that are stored on the host physical machines file system that act as virtualized hard drives for guests. To add file-based storage, perform the following steps:
Pro ced u re 14 .1. Ad d in g f ile- b ased st o rag e
1. Create a storage file or use an existing file (such as an IMG file). Note that both of the following commands create a 4GB file which can be used as additional storage for a guest:
Pre-allocated files are recommended for file-based storage images. Create a pre-allocated file using the following d d command as shown:
# dd if=/dev/zero of=/var/lib/libvirt/images/FileName.img bs=1M count=4096
Alternatively, create a sparse file instead of a pre-allocated file. Sparse files are created much faster and can be used for testing, but are not recommended for production environments due to data integrity and performance issues.
# dd if=/dev/zero of=/var/lib/libvirt/images/FileName.img bs=1M seek=4096 count=0
2. Create the additional storage by writing a <disk> element in a new file. In this example, this file will be known as NewSto rag e. xml .
A <d i sk> element describes the source of the disk, and a device name for the virtual block device. The device name should be unique across all devices in the guest, and identifies the bus on which the guest will find the virtual block device. The following example defines a virtio block device whose source is a file-based storage container named Fi l eName. i mg :
<disk type='file' device='disk'>
152
Chapt er 1 4 . Volumes
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/FileName.img'/>
<target dev='vdb'/>
</disk>
Device names can also start with "hd" or "sd", identifying respectively an IDE and a SCSI disk. The configuration file can also contain an <ad d ress> sub-element that specifies the position on the bus for the new device. In the case of virtio block devices, this should be a
PCI address. Omitting the <ad d ress> sub-element lets libvirt locate and assign the next available PCI slot.
3. Attach the CD-ROM as follows:
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/FileName.img'/>
<readonly/>
<target dev='hdc'/>
</disk >
4. Add the device defined in NewSto rag e. xml with your guest (G uest1):
# virsh attach-device --config Guest1 ~/NewStorage.xml
Note
This change will only apply after the guest has been destroyed and restarted. In addition, persistent devices can only be added to a persistent domain, that is a domain whose configuration has been saved with vi rsh d efi ne command.
If the guest is running, and you want the new device to be added temporarily until the guest is destroyed, omit the --co nfi g option:
# virsh attach-device Guest1 ~/NewStorage.xml
Note
The vi rsh command allows for an attach-d i sk command that can set a limited number of parameters with a simpler syntax and without the need to create an XML file.
The attach-d i sk command is used in a similar manner to the attach-d evi ce command mentioned previously, as shown:
# virsh attach-disk Guest1
/var/lib/libvirt/images/FileName.img vdb --cache none
Note that the vi rsh attach-d i sk command also accepts the --co nfi g option.
5. Start the guest machine (if it is currently not running):
153
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# virsh start Guest1
Note
The following steps are Linux guest specific. Other operating systems handle new storage devices in different ways. For other systems, refer to that operating system's documentation.
6.
Part it io n in g t h e d isk d rive
The guest now has a hard disk device called /d ev/vd b. If required, partition this disk drive and format the partitions. If you do not see the device that you added, then it indicates that there is an issue with the disk hotplug in your guest's operating system.
a. Start fd i sk for the new device:
# fdisk /dev/vdb
Command (m for help): b. Type n for a new partition.
c. The following appears:
Command action e extended p primary partition (1-4)
Type p for a primary partition.
d. Choose an available partition number. In this example, the first partition is chosen by entering 1.
Partition number (1-4): 1 e. Enter the default first cylinder by pressing Enter.
First cylinder (1-400, default 1): f. Select the size of the partition. In this example the entire disk is allocated by pressing
Enter.
Last cylinder or +size or +sizeM or +sizeK (2-400, default
400): g. Enter t to configure the partition type.
Command (m for help): t
154
Chapt er 1 4 . Volumes
h. Select the partition you created in the previous steps. In this example, the partition number is 1 as there was only one partition created and fdisk automatically selected partition 1.
Partition number (1-4): 1 i. Enter 83 for a Linux partition.
Hex code (type L to list codes): 83 j. Enter w to write changes and quit.
Command (m for help): w k. Format the new partition with the ext3 file system.
# mke2fs -j /dev/vdb1
7. Create a mount directory, and mount the disk on the guest. In this example, the directory is located in myfiles.
# mkdir /myfiles
# mount /dev/vdb1 /myfiles
The guest now has an additional virtualized file-based storage device. Note however, that this storage will not mount persistently across reboot unless defined in the guest's
/etc/fstab file:
/dev/vdb1 /myfiles ext3 defaults 0 0
14 .3.2. Adding hard drives and ot her block devices t o a guest
System administrators have the option to use additional hard drives to provide increased storage space for a guest, or to separate system data from user data.
Pro ced u re 14 .2. Ad d in g p h ysical b lo ck d evices t o g u est s
1. This procedure describes how to add a hard drive on the host physical machine to a guest. It applies to all physical block devices, including CD-ROM, DVD and floppy devices.
Physically attach the hard disk device to the host physical machine. Configure the host physical machine if the drive is not accessible by default.
2. Do one of the following: a. Create the additional storage by writing a d i sk element in a new file. In this example, this file will be known as NewSto rag e. xml . The following example is a configuration file section which contains an additional device-based storage container for the host physical machine partition /d ev/sr0 :
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/sr0'/>
<target dev='vdc' bus='virtio'/>
155
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
</disk> b. Follow the instruction in the previous section to attach the device to the guest virtual machine. Alternatively, you can use the virsh attach-disk command, as shown:
# virsh attach-disk Guest1 /dev/sr0 vdc
Note that the following options are available:
The vi rsh attach-d i sk command also accepts the --config, --type, and -
-mode options, as shown:
# vi rsh attach-d i sk G uest1 /d ev/sr0 vd c --co nfi g --type
cd ro m --mo d e read o nl y
Additionally, --type also accepts --type disk in cases where the device is a hard drive.
3. The guest virtual machine now has a new hard disk device called /d ev/vd c on Linux (or something similar, depending on what the guest virtual machine OS chooses) or D : d ri ve
(for example) on Windows. You can now initialize the disk from the guest virtual machine, following the standard procedures for the guest virtual machine's operating system. Refer to
Procedure 14.1, “ Adding file-based storage”
and
Warning
The host physical machine should not use filesystem labels to identify file systems in the fstab file, the i ni trd file or on the kernel command line. Doing so presents a security risk if less privileged users, such as guest virtual machines, have write access to whole partitions or LVM volumes, because a guest virtual machine could potentially write a filesystem label belonging to the host physical machine, to its own block device storage. Upon reboot of the host physical machine, the host physical machine could then mistakenly use the guest virtual machine's disk as a system disk, which would compromise the host physical machine system.
It is preferable to use the UUID of a device to identify it in the fstab file, the i ni trd file or on the kernel command line. While using UUIDs is still not completely secure on certain file systems, a similar compromise with UUID is significantly less feasible.
Important
Guest virtual machines should not be given write access to whole disks or block devices (for example, /d ev/sd b). Guest virtual machines with access to whole block devices may be able to modify volume labels, which can be used to compromise the host physical machine system. Use partitions (for example, /d ev/sd b1) or LVM volumes to prevent this issue.
14 .4 . Delet ing and removing volumes
156
Chapt er 1 4 . Volumes
This section shows how to delete a disk volume from a block based storage pool using the vi rsh
vo l -d el ete command. In this example, the volume is volume 1 and the storage pool is
guest_images.
# virsh vol-delete --pool guest_images volume1
Vol volume1 deleted
157
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 15. Managing guest virtual machines with virsh
vi rsh is a command line interface tool for managing guest virtual machines and the hypervisor. The
vi rsh command-line tool is built on the l i bvi rt management API and operates as an alternative to the q emu-kvm command and the graphical vi rt-manag er application. The vi rsh command can be used in read-only mode by unprivileged users or, with root access, full administration functionality. The vi rsh command is ideal for scripting virtualization administration.
15.1. Generic Commands
The commands in this section are generic because they are not specific to any domain.
15.1.1. help
$ vi rsh hel p [co mmand | g ro up] The help command can be used with or without options. When used without options, all commands are listed, one perline. When used with an option, it is grouped into categories, displaying the keyword for each group.
To display the commands that are only for a specific option, you need to give the keyword for that group as an option. For example:
$ vi rsh hel p po o l
Storage Pool (help keyword 'pool'):
find-storage-pool-sources-as find potential storage pool sources
find-storage-pool-sources discover potential storage pool sources
pool-autostart autostart a pool
pool-build build a pool
pool-create-as create a pool from a set of args
pool-create create a pool from an XML file
pool-define-as define a pool from a set of args
pool-define define (but don't start) a pool from an XML file
pool-delete delete a pool
pool-destroy destroy (stop) a pool
pool-dumpxml pool information in XML
pool-edit edit XML configuration for a storage pool
pool-info storage pool information
pool-list list pools
pool-name convert a pool UUID to pool name
pool-refresh refresh a pool
pool-start start a (previously defined) inactive pool
pool-undefine undefine an inactive pool
pool-uuid convert a pool name to pool UUID
Using the same command with a command option, gives the help information on that one specific command. For example:
$vi rsh hel p vo l -path
NAME
vol-path - returns the volume path for a given volume name or key
158
Chapt er 1 5. Managing guest virt ual machines wit h virsh
SYNOPSIS
vol-path <vol> [--pool <string>]
OPTIONS
[--vol] <string> volume name or key
--pool <string> pool name or uuid
15.1.2. quit and exit
The quit command and the exit command will close the terminal. For example:
$vi rsh exi t
$vi rsh q ui t
15.1.3. version
The version command displays the current libvirt version and displays information about where the build is from. For example:
$ vi rsh versi o n
Compiled against library: libvirt 1.1.1
Using library: libvirt 1.1.1
Using API: QEMU 1.1.1
Running hypervisor: QEMU 1.5.3
15.1.4 . Argument display
vi rsh echo [--shel l ][--xml ][arg ] echos or displays the specified argument. Each argument echoed will be separated by a space. by using the --shel l argument, the output will be single quoted where needed so that it is suitable for reusing in a shell command. if --xml argument is used the output will be made suitable for use in an XML file.
15.1.5. connect
Connects to a hypervisor session. When the shell is first started this command runs automatically when the URI parameter is requested by the -c command. The URI specifies how to connect to the hypervisor. The most commonly used URIs are:
xen: ///- connects to the local XEN hypervisor
q emu: ///system - connects locally as root to the daemon supervising QEMU and KVM domains.
xen: ///sessi o n - connects locally as a user to the user's set of QEMU and KVM domains
l xc: /// - connects to a local Linux container
Additional values are available on libvert's website http://libvirt.org/uri.html
The command can be run as follows:
$vi rsh co nnect {name|URI}
159
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Where {name} is the machine name (hostname) or URL (the output of the vi rsh uri command) of the hypervisor. To initiate a read-only connection, append the above command with --read o nl y.
For more information on URIs refer to Remote URIs If you are unsure of the URI, the uri command will display it:
$ vi rsh uri qemu:///session
15.1.6. Displaying basic informat ion
The following commands may be used to display basic information:
$ ho stname - displays the hypervisor's hostname
$ sysi nfo - displays the XML representation of the hypervisor's system information, if available
15.1.7. Inject ing NMI
The $ vi rsh i nject-nmi [d o mai n] injects NMI (non-maskable interrupt) message to the guest virtual machine. This is used when response time is critical, such as non-recoverable hardware errors. To run this command:
$ virsh inject-nmi guest-1
15.2. At t aching and updat ing a device wit h virsh
Pro ced u re 15.1. Ho t p lu g g in g USB d evices f o r u se b y t h e g u est virt u al mach in e
The following procedure demonstrates how to attach USB devices to the guest virtual machine. This can be done while the guest virtual machine is running as a hotplug procedure or it can be done while the guest is shutoff. The device you want to emulate needs to be attached to the host physical machine.
1. Locate the USB device you want to attach with the following command:
# lsusb -v idVendor 0x17ef Lenovo idProduct 0x480f Integrated Webcam [R5U877]
2. Create an XML file and give it a logical name (usb_d evi ce. xml , for example). Make sure you copy the vendor and procuct IDs exactly as was displayed in your search.
160
<hostdev mode= 'subsystem' type= 'usb' managed= 'yes' >
<source>
Chapt er 1 5. Managing guest virt ual machines wit h virsh
<vendor id= '0x17ef' />
<product id= '0x480f' />
</source>
</hostdev>
...
Fig u re 15.1. USB Devices XML Sn ip p et
3. Attach the device with the following command:
# virsh attach-device rhel7 --fi l e usb_d evi ce. xml > --co nfi g
In this example [rhel7] is the name of your guest virtual machine and [usb_device.xml] is the file you created in the previous step. If you want to have the change take effect in the next reboot, use the --co nfi g If you want this change to be persistent, use the --persi stent argument. If you want the change to take effect on the current domain, use the --current argument. See the Virsh MAN page for additional arguments.
4. If you want to detach the device (hot unplug), perform the following command:
# virsh detach-device rhel7 --fi l e usb_d evi ce. xml >
In this example [rhel7] is the name of your guest virtual machine and [usb_device.xml] is the file you attached in the previous step
15.3. At t aching int erface devices
The vi rsh attach-i nterfacedomain type source command can take the following arguments:
--l i ve - get value from running domain
--co nfi g - get value to be used on next boot
--current - get value according to current domain state
--persi stent - behaves like --co nfi g for an offline domain, and like --l i ve for a running domain.
--targ et - indicates the target device in the guest virtual machine.
--mac - use this to specify the MAC address of the network interface
--scri pt - use this to specify a path to a script file handling a bridge instead of the default one.
--mo d el - use this to specify the model type.
--i nbo und - controls the inbound bandwidth of the interface. Acceptable values are averag e,
peak, and burst.
--o utbo und - controls the outbound bandwidth of the interface. Acceptable values are
averag e, peak, and burst.
161
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The type can be either netwo rk to indicate a physical network device, or bri d g e to indicate a bridge to a device. source is the source of the device. To remove the attached device, use the vi rsh
d etach-d evi ce.
15.4 . Changing t he media of a CDROM
Changing the media of a CDROM to another source or format
# chang e-med i a domain path source --eject --i nsert --upd ate --current --
l i ve --co nfi g --fo rce
--path - A string containing a fully-qualified path or target of disk device
--so urce - A string containing the source of the media
--eject - Eject the media
--i nsert - Insert the media
--upd ate - Update the media
--current - can be either or both of --l i ve and --co nfi g , depends on implementation of hypervisor driver
--l i ve - alter live configuration of running domain
--co nfi g - alter persistent configuration, effect observed on next boot
--fo rce - force media changing
15.5. Domain Commands
A domain name is required for most of these commands as they manipulate the specified domain directly. The domain may be given as a short integer (0,1,2...), a name, or a full UUID.
15.5.1. Configuring a domain t o be st art ed aut omat ically at boot
$ vi rsh auto start [--d i sabl e] d o mai n will automatically start the specified domain at boot.
Using the --d i sabl e argument disables autostart.
# vi rsh auto start rhel7
In the example above, the rhel7 guest virtual machine will automatically start when the host physical machine boots
# vi rsh auto start rhel7--d i sabl e
In the example above, the autostart function is disabled and the guest virtual machine will no longer start automatically when the host physical machine boots.
15.5.2. Connect ing t he serial console for t he guest virt ual machine
The $ vi rsh co nso l e <d o mai n> [--d evname <stri ng >] [--fo rce] [--safe] command connects the virtual serial console for the guest virtual machine. The optional --devname
162
Chapt er 1 5. Managing guest virt ual machines wit h virsh
<string> parameter refers to the device alias of an alternate console, serial, or parallel device configured for the guest virtual machine. If this parameter is omitted, the primary console will be opened. The --fo rce argument will force the console connection or when used with disconnect, will disconnect connections. Using the --safe argument will only allow the guest to connect if safe console handling is supported.
$ virsh console virtual_machine --safe
15.5.3. Defining a domain wit h an XML file
The d efi ne <FILE> command defines a domain from an XML file. The domain definition in this case is registered but not started. If the domain is already running, the changes will take effect on the next boot.
15.5.4 . Edit ing and displaying a descript ion and t it le of a domain
Note that the information you enter in this command does not in any way This command is only used to show or modify description and title of a domain, but ford not configure it at all. These values are merely user fields that allow storage of arbitrary textual data to allow easy identification of domains.
Ideally, the title should be short, although it’s not enforced.
The arguments --l i ve or --co nfi g select whether this command works on live or persistent definitions of the domain. If both --l i ve and --co nfi g are specified, the --co nfi g option takes precedence on getting the current description and both live configuration and persistent configuration are updated while setting the description. --current argument will modify or get the current state configuration. It is exclusive and implied if none of these were specified. The --ed i t argument specifies that an editor with the contents of current description or title should be opened and the contents saved back afterwards. --title selects operation on the title field instead of a description. In addition, if neither --ed i t nor --new-d esc are specified, then the description is displayed and cannot be modified.
15.5.5. Displaying device block st at ist ics
This command will display the block statistics for a running domain. You need to have both the domain name and the device name (use the vi rsh d o mbl kl i st to list the devices.)In this case a block device is the unique target name (<target dev='name'/>) or a source file (< source file
='name'/>). Note that not every hypervisor can display every field. To make sure that the output is presented in its most legible form use the --human argument, as shown:
# virsh domblklist rhel7
Target Source
-----------------------------------------------vda /VirtualMachines/rhel7.img
hdc -
# virsh domblkstat --human rhel7 vda
Device: vda
number of read operations: 174670
number of bytes read: 3219440128
number of write operations: 23897
number of bytes written: 164849664
163
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
number of flush operations: 11577
total duration of reads (ns): 1005410244506
total duration of writes (ns): 1085306686457
total duration of flushes (ns): 340645193294
15.5.6. Ret rieving net work st at ist ics
The d o mnetstat [d o mai n][i nterface-d evi ce] command displays the network interface statistics for the specified device running on a given domain.
# domifstat rhel7 eth0
15.5.7. Modifying t he link st at e of a domain's virt ual int erface
This command can either configure a specified interface as up or down. The d o mi f-setl i nk
[d o mai n][i nterface-d evi ce][state]{--co nfi g } modifies the status of the specified interface for the specified domain. Note that if you only want the persistent configuration of the domain to be modified, you need to use the --co nfi g argument. It should also be noted that for compatibility reasons, --persi stent is an alias of --co nfi g . The "interface device" can be the interface's target name or the MAC address.
# d o mi f-setl i nk rhel 7 eth0 up
15.5.8. List ing t he link st at e of a domain's virt ual int erface
This command can be used to query the state of a specified interface on a given domain. Note that if you only want the persistent configuration of the domain to be modified, you need to use the --
co nfi g argument. It should also be noted that for compatibility reasons, --persi stent is an alias of --co nfi g . The "interface device" can be the interface's target name or the MAC address.
# d o mi f-g etl i nk rhel 7 eth0 up
15.5.9. Set t ing net work int erface bandwidt h paramet ers
d o mi ftune sets the guest virtual machine's network interface bandwidth parameters. The following format should be used:
#vi rsh d o mi ftune d o mai n i nterface-d evi ce [[--co nfi g ] [--l i ve] | [--
current]] [--i nbo und averag e,peak,burst] [--o utbo und averag e,peak,burst]
The only required parameter is the domain name and interface device of the guest virtual machine,
limits with the following flags:
<interface-device> This is mandatory and it will set or query the domain’s network interface’s bandwidth parameters. i nterface-d evi ce can be the interface’s target name (<target dev=’name’/>), or the MAC address.
If no --i nbo und or --o utbo und is specified, this command will query and show the bandwidth settings. Otherwise, it will set the inbound or outbound bandwidth. average,peak,burst is the same as in attach-i nterface command. Refer to
Section 15.3, “ Attaching interface devices”
164
Chapt er 1 5. Managing guest virt ual machines wit h virsh
15.5.10. Ret rieving memory st at ist ics for a running domain
This command may return varied results depending on the hypervisor you are using.
The d o mmemstat [d o mai n] [--peri o d (sec)][[--co nfi g ][--l i ve]| [--current]] displays the memory statistics for a running domain. Using the --peri o d argument requires a time period in seconds. Setting this argument to a value larger than 0 will allow the balloon driver to return additional statistics which will be displayed by subsequent d o memstat commands. Setting the --peri o d argument to 0, will stop the balloon driver collection but does not clear the statistics in the balloon driver. You cannot use the --l i ve, --co nfi g , or --current arguments without also setting the --peri o d option in order to also set the collection period for the balloon driver. If the --
l i ve argument is specified, only the running guest's collection period is affected. If the --
co nfi g argument is used, it will affect the next boot of a persistent guest. If --current argument is used, it will affect the current guest state
Both the --l i ve and --co nfi g arguments may be used but --current is exclusive. If no flag is specified, the behavior will be different depending on the guest's state.
#vi rsh d o memstat rhel7--current
15.5.11. Displaying errors on block devices
This command is best used following a d o mstate that reports that a domain is paused due to an I/O error. The d o mbl kerro r domain command shows all block devices that are in error state on a given domain and it displays the error message that the device is reporting.
# vi rsh d o mbl kerro r rhel7
15.5.12. Displaying t he block device siz e
In this case a block device is the unique target name (<target dev='name'/>) or a source file (< source file ='name'/>). To retrieve a list you can run d o mbl kl i st. This d o mbl ki nfo requires a domain name.
# vi rsh d o mbl ki nfo rhel7
15.5.13. Displaying t he block devices associat ed wit h a domain
The d o mbl kl i st domain --i nacti ve--d etai l s displays a table of all block devices that are associated with the specified domain.
If --i nacti ve is specified, the result will show the devices that are to be used at the next boot and will not show those that are currently running in use by the running domain. If --d etai l s is specified, the disk type and device value will be included in the table. The information displayed in this table can be used with the d o mbl ki nfo and snapsho t-create.
#d o mbl kl i st rhel7 --d etai l s
15.5.14 . Displaying virt ual int erfaces associat ed wit h a domain
Running the d o mi fl i st command results in a table that displays information of all the virtual interfaces that are associated with a specified domain. The d o mi fl i st requires a domain name and optionally can take the --i nacti ve argument.
165
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
If --i nacti ve is specified, the result will show the devices that are to be used at the next boot and will not show those that are currently running in use by the running domain.
Commands that require a MAC address of a virtual interface (such as d etach-i nterface or
d o mi f-setl i nk) will accept the output displayed by this command.
15.5.15. Using blockcommit t o short en a backing chain
This section demonstrates how to use bl o ckco mmi t to shorten a backing chain. For more
background on backing chains, see Section 15.5.18, “ Disk image management with live block copy”
.
bl o ckco mmi t copies data from one part of the chain down into a backing file, allowing you to pivot the rest of the chain in order to bypass the committed portions. For example, suppose this is the current state:
base ← snap1 ← snap2 ← acti ve.
Using bl o ckco mmi t moves the contents of snap2 into snap1, allowing you to delete snap2 from the chain, making backups much quicker.
Pro ced u re 15.2. virsh b lo ckco mmit
Run the following command:
# vi rsh bl o ckco mmi t $d o m $d i sk -base snap1 -to p snap2 -wai t -verbo se
The contents of snap2 are moved into snap1, resulting in:
base ← snap1 ← acti ve. Snap2 is no longer valid and can be deleted
Warning
bl o ckco mmi t will corrupt any file that depends on the -base argument (other than files that depended on the -top argument, as those files now point to the base). To prevent this, do not commit changes into files shared by more than one guest. The -verbose option will allow the progress to be printed on the screen.
15.5.16. Using blockpull t o short en a backing chain
bl o ckpul l can be used in in the following applications:
Flattens an image by populating it with data from its backing image chain. This makes the image file self-contained so that it no longer depends on backing images and looks like this:
Before: base.img ← Active
After: base.img is no longer used by the guest and Active contains all of the data.
Flattens part of the backing image chain. This can be used to flatten snapshots into the top-level image and looks like this:
Before: base ← sn1 ←sn2 ← active
166
Chapt er 1 5. Managing guest virt ual machines wit h virsh
After: base.img ← active. Note that active now contains all data from sn1 and sn2 and neither sn1 nor sn2 are used by the guest.
Moves the disk image to a new file system on the host. This is allows image files to be moved while the guest is running and looks like this:
Before (The original image file): /fs1/base. vm. i mg
After: /fs2/acti ve. vm. q co w2 is now the new file system and /fs1/base. vm. i mg is no longer used.
Useful in live migration with post-copy storage migration. The disk image is copied from the source host to the destination host after live migration completes.
In short this is what happens: Before:/so urce-ho st/base. vm. i mg After:/d esti nati o n-
ho st/acti ve. vm. q co w2./so urce-ho st/base. vm. i mg is no longer used.
Pro ced u re 15.3. Usin g b lo ckp u ll t o sh o rt en a b ackin g ch ain
1. It may be helpful to run this command prior to running bl o ckpul l :
# vi rsh snapsho t-create-as $d o m $name - d i sk-o nl y
2. If the chain looks like this: base ← snap1 ← snap2 ← acti ve run the following:
# vi rsh bl o ckpul l $d o m $d i sk snap1
This command makes 'snap1' the backing file of active, by pulling data from snap2 into active resulting in: base ← snap1 ← active.
3. Once the bl o ckpul l is complete, the lib virt tracking of the snapshot that created the extra image in the chain is no longer useful. Delete the tracking on the outdated snapshot with this command:
# vi rsh snapsho t-d el ete $d o m $name - metad ata
Additional applications of bl o ckpul l can be done as follows:
To flatten a single image and populate it with data from its backing image chain:# vi rsh
bl o ckpul l exampl e-d o mai n vd a - wai t
To flatten part of the backing image chain:# vi rsh bl o ckpul l exampl e-d o mai n vd a -
base /path/to /base. i mg - wai t
To move the disk image to a new file system on the host:# vi rsh snapsho t-create
exampl e-d o mai n - xml fi l e /path/to /new. xml - d i sk-o nl y followed by # vi rsh
bl o ckpul l exampl e-d o mai n vd a - wai t
To use live migration with post-copy storage migration:
On the destination run:
# q emu-i mg create -f q co w2 -o backi ng _fi l e= /so urce-ho st/vm. i mg
/d esti nati o n-ho st/vm. q co w2
On the source run:
167
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# vi rsh mi g rate exampl e-d o mai n
On the destination run:
# vi rsh bl o ckpul l exampl e-d o mai n vd a - wai t
15.5.17. Using blockresiz e t o change t he siz e of a domain pat h
bl o ckresi ze can be used to re-size a block device of a domain while the domain is running, using the absolute path of the block device which also corresponds to a unique target name (<targ et
d ev= "name"/>) or source file (<so urce fi l e= "name"/>). This can be applied to one of the disk devices attached to domain (you can use the command d o mbl kl i st to print a table showing the brief information of all block devices associated with a given domain).
Note
Live image re-sizing will always re-size the image, but may not immediately be picked up by guests. With recent guest kernels, the size of virtio-blk devices is automatically updated (older kernels require a guest reboot). With SCSI devices, it is required to manually trigger a re-scan in the guest with the command, echo >
/sys/cl ass/scsi _d evi ce/0 : 0 : 0 : 0 /d evi ce/rescan. In addition, with IDE it is required to reboot the guest before it picks up the new size.
Run the following command: bl o ckresi ze [d o mai n] [path si ze] where:
Domain is the unique target name or source file of the domain whose size you want to change
Path size is a scaled integer which defaults to KiB (blocks of 1024 bytes) if there is no suffix.
You must use a suffix of "B" to for bytes.
15.5.18. Disk image management wit h live block copy
Note
Live block copy is a feature that is not supported with the version of KVM that is supplied with
Red Hat Enterprise Linux. Live block copy is available with the version of KVM that is supplied with Red Hat Virtualization. This version of KVM must be running on your physical host machine in order for the feature to be supported. Contact your representative at Red Hat for more details.
Live block copy allows you to copy an in use guest disk image to a destination image and switches the guest disk image to the destination guest image while the guest is running. Whilst live migration moves the memory and registry state of the host, the guest is kept in shared storage. Live block copy allows you to move the entire guest contents to another host on the fly while the guest is running. Live block copy may also be used for live migration without requiring permanent share storage. In this method the disk image is copied to the destination host after migration, but while the guest is running.
Live block copy is especially useful for the following applications: moving the guest image from local storage to a central location
168
Chapt er 1 5. Managing guest virt ual machines wit h virsh
when maintenance is required, guests can be transferred to another location, with no loss of performance allows for management of guest images for speed and efficiency image format conversions can be done without having to shut down the guest
Examp le 15.1. Examp le u sin g live b lo ck co p y
This example shows what happens when live block copy is performed. The example has a backing file (base) that is shared between a source and destination. It also has two overlays (sn1 and sn2) that are only present on the source and must be copied.
1. The backing file chain at the beginning looks like this:
base ← sn1 ← sn2
The components are as follows: base - the original disk image sn1 - the first snapshot that was taken of the base disk image sn2 - the most current snapshot active - the copy of the disk
2. When a copy of the image is created as a new image on top of sn2 the result is this:
base ← sn1 ← sn2 ← acti ve
3. At this point the read permissions are all in the correct order and are set automatically. To make sure write permissions are set properly, a mirror mechanism redirects all writes to both sn2 and active, so that sn2 and active read the same at any time (and this mirror mechanism is the essential difference between live block copy and image streaming).
4. A background task that loops over all disk clusters is executed. For each cluster, there are the following possible cases and actions:
The cluster is already allocated in active and there is nothing to do.
Use bd rv_i s_al l o cated () to follow the backing file chain. If the cluster is read from base (which is shared) there is nothing to do.
If bd rv_i s_al l o cated () variant is not feasible, rebase the image and compare the read data with write data in base in order to decide if a copy is needed.
In all other cases, copy the cluster into acti ve
5. When the copy has completed, the backing file of active is switched to base (similar to rebase)
To reduce the length of a backing chain after a series of snapshots, the following commands are
15.5.19. Displaying a URI for connect ion t o a graphical display
169
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Running the vi rsh d o md i spl ay command will output a URI which can then be used to connect to the graphical display of the domain via VNC, SPICE, or RDP. If the argument --i ncl ud e-
passwo rd is used, the SPICE channel password will be included in the URI.
15.5.20. Domain Ret rieval Commands
The following commands will display different information about a given domain
vi rsh d o mho stname domain displays the hostname of the specified domain provided the hypervisor can publish it.
vi rsh d o mi nfo domain displays basic information about a specified domain.
vi rsh d o mui d domain|ID converts a given domain name or ID into a UUID.
vi rsh d o mi d domain|ID converts a given domain name or UUID into an ID.
vi rsh d o mjo babo rt domain aborts the currently running job on the specified domain.
vi rsh d o mjo bi nfo domain displays information about jobs running on the specified domain, including migration statistics
vi rsh d o mname domain ID|UUID converts a given domain ID or UUID into a domain name.
vi rsh d o mstate domain displays the state of the given domain. Using the --reaso n argument will also display the reason for the displayed state.
vi rsh d o mco ntro l domain displays the state of an interface to VMM that were used to control a domain. For states that are not OK or Error, it will also print the number of seconds that have elapsed since the control interface entered the displayed state.
Examp le 15.2. Examp le o f st at ist ical f eed b ack
In order to get information about the domain, run the following command:
# virsh domjobinfo rhel7
Job type: Unbounded
Time elapsed: 1603 ms
Data processed: 47.004 MiB
Data remaining: 658.633 MiB
Data total: 1.125 GiB
Memory processed: 47.004 MiB
Memory remaining: 658.633 MiB
Memory total: 1.125 GiB
Constant pages: 114382
Normal pages: 12005
Normal data: 46.895 MiB
Expected downtime: 0 ms
Compression cache: 64.000 MiB
Compressed data: 0.000 B
Compressed pages: 0
Compression cache misses: 12005
Compression overflows: 0
15.5.21. Convert ing QEMU argument s t o domain XML
170
Chapt er 1 5. Managing guest virt ual machines wit h virsh
The vi rsh d o mxml -fro m-nati ve provides a way to convert an existing set of QEMU arguments into a guest description using libvirt Domain XML that can then be used by libvirt. Please note that this command is intended to be used only to convert existing qemu guests previously started from the command line in order to allow them to be managed through libvirt. The method described here should not be used to create new guests from scratch. New guests should be created using either virsh or virt-manager. Additional information can be found here .
Suppose you have a QEMU guest with the following args file:
$ cat demo.args
LC_ALL=C
PATH=/bin
HOME=/home/test
USER=test
LOGNAME=test /usr/bin/qemu -S -M pc -m 214 -smp 1 -nographic -monitor pty
-no-acpi -boot c -hda /dev/HostVG/QEMUGuest1 -net none -serial none parallel none -usb
To convert this to a domain XML file so that the guest can be managed by libvirt, run:
$ vi rsh d o mxml -fro m-nati ve q emu-arg v d emo . arg s
This command turns the args file above, into this domain XML file:
<domain type='qemu'>
<uuid>00000000-0000-0000-0000-000000000000</uuid>
<memory>219136</memory>
<currentMemory>219136</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='i686' machine='pc'>hvm</type>
<boot dev='hd'/>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu</emulator>
<disk type='block' device='disk'>
<source dev='/dev/HostVG/QEMUGuest1'/>
<target dev='hda' bus='ide'/>
</disk>
</devices>
</domain>
15.5.22. Creat ing a dump file of a domain's core
Sometimes it is necessary (especially in the cases of troubleshooting), to create a dump file containing the core of the domain so that it can be analyzed. In this case, running vi rsh d ump
domain corefilepath --bypass-cache --l i ve | --crash | --reset --verbo se --
memo ry-o nl y dumps the domain core to a file specified by the corefilepath Note that some
171
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
hypervisors may gave restrictions on this action and may require the user to manually ensure proper permissions on the file and path specified in the corefilepath parameter. This command is supported with SR-IOV devices as well as other passthrough devices. The following arguments are supported and have the following effect:
--bypass-cache the file saved will not contain the file system cache. Note that selecting this option may slow down dump operation.
--l i ve will save the file as the domain continues to run and will not pause or stop the domain.
--crash puts the domain in a crashed status rather than leaving it in a paused state while the dump file is saved.
--reset once the dump file is successfully saved, the domain will reset.
--verbo se displays the progress of the dump process
--memo ry-o nl y the only information that will be saved in the dump file will be the domain's memory and CPU common register file.
Note that the entire process can be monitored using the d o mjo bi nfo command and can be canceled using the d o mjo babo rt command.
15.5.23. Creat ing a virt ual machine XML dump (configurat ion file)
Output a guest virtual machine's XML configuration file with vi rsh:
# virsh dumpxml {guest-id, guestname or uuid}
This command outputs the guest virtual machine's XML configuration file to standard out (std o ut).
You can save the data by piping the output to a file. An example of piping the output to a file called
guest.xml:
# virsh dumpxml GuestID > guest.xml
devices or to deploy additional guest virtual machines.
An example of vi rsh d umpxml output:
# virsh dumpxml guest1-rhel7-64
<domain type='kvm'>
<name>guest1-rhel6-64</name>
<uuid>b8d7388a-bbf2-db3a-e962-b97ca6e514bd</uuid>
<memory>2097152</memory>
<currentMemory>2097152</currentMemory>
<vcpu>2</vcpu>
<os>
<type arch='x86_64' machine='rhel6.2.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
172
Chapt er 1 5. Managing guest virt ual machines wit h virsh
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none' io='threads'/>
<source file='/home/guest-images/guest1-rhel6-64.img'/>
<target dev='vda' bus='virtio'/>
<shareable/<
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</disk>
<interface type='bridge'>
<mac address='52:54:00:b9:35:a9'/>
<source bridge='br0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'/>
<sound model='ich6'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</sound>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</memballoon>
</devices>
</domain>
Note that the <shareable/> flag is set. This indicates the device is expected to be shared between domains (assuming the hypervisor and OS support this), which means that caching should be deactivated for that device.
15.5.24 . Creat ing a guest virt ual machine from a configurat ion file
173
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Guest virtual machines can be created from XML configuration files. You can copy existing XML from
previously created guest virtual machines or use the d umpxml option (refer to Section 15.5.23,
vi rsh from an XML file:
# virsh create configuration_file.xml
15.6. Edit ing a guest virt ual machine's configurat ion file
are offline. The vi rsh ed i t command provides this functionality. For example, to edit the guest virtual machine named rhel7:
# virsh edit rhel6
This opens a text editor. The default text editor is the $ED IT O R shell parameter (set to vi by default).
15.6.1. Adding mult ifunct ion PCI devices t o KVM guest virt ual machines
This section will demonstrate how to add multi-function PCI devices to KVM guest virtual machines.
1. Run the vi rsh ed i t [guestname] command to edit the XML configuration file for the guest virtual machine.
2. In the address type tag, add a mul ti functi o n= ' o n' entry for functi o n= ' 0 x0 ' .
This enables the guest virtual machine to use the multifunction PCI devices.
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/rhel62-1.img'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/
</disk>
For a PCI device with two functions, amend the XML configuration file to include a second device with the same slot number as the first device and a different function number, such as
functi o n= ' 0 x1' .
For Example:
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/rhel62-1.img'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/rhel62-2.img'/>
174
Chapt er 1 5. Managing guest virt ual machines wit h virsh
<target dev='vdb' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
</disk>
3. l spci output from the KVM guest virtual machine shows:
$ lspci
00:05.0 SCSI storage controller: Red Hat, Inc Virtio block device
00:05.1 SCSI storage controller: Red Hat, Inc Virtio block device
15.6.2. St opping a running domain in order t o rest art it lat er
vi rsh manag ed save domain --bypass-cache --runni ng | --paused | --verbo se
saves and destroys (stops) a running domain so that it can be restarted from the same state at a later time. When used with a vi rsh start command it is automatically started from this save point. If it is used with the --bypass-cache argument the save will avoid the filesystem cache. Note that this option may slow down the save process speed.
--verbo se displays the progress of the dump process
Under normal conditions, the managed save will decide between using the running or paused state as determined by the state the domain is in when the save is done. However, this can be overridden by using the --runni ng argument to indicate that it must be left in a running state or by using --
paused argument which indicates it is to be left in a paused state.
To remove the managed save state, use the vi rsh manag ed save-remo ve command which will force the domain to do a full boot the next time it is started.
Note that the entire managed save process can be monitored using the d o mjo bi nfo command and can also be canceled using the d o mjo babo rt command.
15.6.3. Displaying CPU st at ist ics for a specified domain
The vi rsh cpu-stats domain --to tal start co unt command provides the CPU statistical information on the specified domain. By default it shows the statistics for all CPUs, as well as a total.
The --to tal argument will only display the total statistics.
15.6.4 . Saving a screenshot
The vi rsh screensho t command takes a screenshot of a current domain console and stores it into a file. If however the hypervisor supports more displays for a domain, using the --screen and giving a screen ID will specify which screen to capture. In the case where there are multiple graphics cards, where the heads are numerated before their devices, screen ID 5 addresses the second head on the second card.
15.6.5. Sending a keyst roke combinat ion t o a specified domain
Using the vi rsh send -key domain --co d eset --ho l d ti me keycode command you can send a sequence as a keycode to a specific domain.
175
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Each keycode can either be a numeric value or a symbolic name from the corresponding codeset. If multiple keycodes are specified, thay are all sent simultaneously to the guest virtual machine and as such may be received in random order. If you need distinct keycodes, you must send the send -key command multiple times.
# vi rsh send -key rhel6 --ho l d ti me 10 0 0 0xf
If a --ho l d ti me is given, each keystroke will be held for the specified amount in milliseconds. The -
-co d eset allows you to specify a code set, the default being Linux, but the following options are permitted:
l i nux - choosing this option causes the symbolic names to match the corresponding Linux key constant macro names and the numeric values are those offered by the Linux generic input event subsystems.
xt- this will send a value that is defined by the XT keyboard controller. No symbolic names are provided.
atset1 - the numeric values are those that are defined by the AT keyboard controller, set1 (XT compatible set). Extended keycodes from the atset1 may differ from extended keycodes in the XT codeset. No symbolic names are provided.
atset2 - The numeric values are those defined by the AT keyboard controller, set 2. No symbolic names are provided.
atset3 - The numeric values are those defined by the AT keyboard controller, set 3 (PS/2 compatible). No symbolic names are provided.
o s_x - The numeric values are those defined by the OS-X keyboard input subsystem. The symbolic names match the corresponding OS-X key constant macro names.
xt_kbd - The numeric values are those defined by the Linux KBD device. These are a variant on the original XT codeset, but often with different encoding for extended keycodes. No symbolic names are provided.
wi n32 - The numeric values are those defined by the Win32 keyboard input subsystem. The symbolic names match the corresponding Win32 key constant macro names.
usb - The numeric values are those defined by the USB HID specification for keyboard input. No symbolic names are provided.
rfb - The numeric values are those defined by the RFB extension for sending raw keycodes.
These are a variant on the XT codeset, but extended keycodes have the low bit of the second bite set, instead of the high bit of the first byte. No symbolic names are provided.
15.6.6. Sending process signal names t o virt ual processes
Using the vi rsh send -pro cess-si g nal domain-ID PID signame you can send a signal signame to a specified virtual process (as identified by its Process ID or PID) within a running domain given it's domain ID. In addition to an integer signal constant number, one or more of the following signames can be sent:
no p , stkfl t
hup, co nt
i nt, chl d
176
Chapt er 1 5. Managing guest virt ual machines wit h virsh
q ui t, sto p
i l l , tstp
trap, tti n
abrt, tto u
bus, urg
fpe, xcpu
ki l l , xfsz
usr1, vtal rm
seg v, pro f
usr2, wi nch
pi pe, po l l
al rm, pwr
term, sys
More options are on the Virsh MAN page. Note that these sumbols may also be prefixed with
si g or si g _ and it is not case sensitive.
# vi rsh send -pro cess-si g nal rhel 7 187 ki l l
15.6.7. Displaying t he IP address and port number for t he VNC display
The vi rsh vncd i spl ay will print the IP address and port number of the VNC display for the specified domain. If the information is unavailable the exit code 1 will be displayed.
# virsh vncdisplay rhel7
127.0.0.1:0
15.7. NUMA node management
This section contains the commands needed for NUMA node management.
15.7.1. Displaying node informat ion
The no d ei nfo command displays basic information about the node, including the model number, number of CPUs, type of CPU, and size of the physical memory. The output corresponds to
vi rNo d eInfo structure. Specifically, the "CPU socket(s)" field indicates the number of CPU sockets per NUMA cell.
$ vi rsh no d ei nfo
CPU model: x86_64
CPU(s): 4
CPU frequency: 1199 MHz
CPU socket(s): 1
177
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Core(s) per socket: 2
Thread(s) per core: 2
NUMA cell(s): 1
Memory size: 3715908 KiB
15.7.2. Set t ing NUMA paramet ers
The vi rsh numatune can either set or retrieve the NUMA parameters for a specified domain. Within the Domain XML file these parameters are nested within the <numatune> element. Without using flags, only the current settings are displayed. The numatune domain command requires a specified domain and can take the following arguments:
--mo d e - The mode can be set to either stri ct, i nterl eave, or preferred . Running domains cannot have their mode changed while live unless the domain was started within stri ct mode.
--no d eset contains a list of NUMA nodes that are used by the host physical machine for running the domain. The list contains nodes, each separated by a comma, with a dash - used for node ranges and a caret ^ used for excluding a node.
Only one of the three following flags can be used per instance
--co nfi g will effect the next boot of a persistent guest virtual machine
--l i ve will set the scheduler information of a running guest virtual machine.
--current will effect the current state of the guest virtual machine.
15.7.3. Displaying t he amount of free memory in a NUMA cell
The vi rsh freecel l displays the available amount of memory on the machine within a specified
NUMA cell. This command can provide one of three different displays of available memory on the machine depending on the options specified. If no options are used, the total free memory on the machine is displayed. Using the --al l option, it displays the free memory in each cell and the total free memory on the machine. By using a numeric argument or with --cel l no along with a cell number it will display the free memory for the specified cell.
15.7.4 . Displaying a CPU list
The no d ecpumap command displays the number of CPUs that are available to the node, whether they are online or not and it also lists the number that are currently online.
$ vi rsh no d ecpumap
CPUs present: 4
CPUs online: 1
CPU map: y
15.7.5. Displaying CPU st at ist ics
The no d ecpustats command displays statistical information about the specified CPU, if the CPU is given. If not, it will display the CPU status of the node. If a percent is specified, it will display the percentage of each type of CPU statistics that were recorded over an one (1) second interval.
This example shows no CPU specified:
$ vi rsh no d ecpustats
178
Chapt er 1 5. Managing guest virt ual machines wit h virsh
user: 1056442260000000 system: 401675280000000 idle: 7549613380000000 iowait: 94593570000000
This example shows the statistical percentages for CPU number 2:
$ vi rsh no d ecpustats 2 --percent usage: 2.0% user: 1.0% system: 1.0% idle: 98.0% iowait: 0.0%
You can control the behavior of the rebooting guest virtual machine by modifying the o n_rebo o t element in the guest virtual machine's configuration file.
15.7.6. Suspending t he host physical machine
The no d esuspend command puts the host physical machine into a system-wide sleep state similar to that of Suspend-to-RAM (s3), Suspend-to-Disk (s4), or Hybrid-Suspend and sets up a Real-Time-
Clock to wake up the node after the duration that is set has past. The --targ et argument can be set to either mem,disk, or hybrid. These options indicate to set the memory, disk, or combination of the two to suspend. Setting the --d urati o n instructs the host physical machine to wake up after the set duration time has run out. It is set in seconds. It is recommended that the duration time be longer than
60 seconds.
$ vi rsh no d esuspend d i sk 6 0
15.7.7. Set t ing and displaying t he node memory paramet ers
The no d e-memo ry-tune [shm-pag es-to -scan] [shm-sl eep-mi l i secs] [shm-merg e-
acro ss-no d es] command displays and allows you to set the node memory parameters. There are three parameters that may be set with this command:
shm-pag es-to -scan - sets the number of pages to scan before the shared memory service goes to sleep.
shm-sl eep-mi l i secs - sets the number of miliseconds that the shared memory service will sleep before the next scan
shm-merg e-acro ss-no d es - specifies if pages from different NUMA nodes can be merged.
Values allowed are 0 and 1. When set to 0, the only pages that can be merged are those that are physically residing in the memory area of the same NUMA node. When set to 1, pages from all of the NUMA nodes can be merged. The default setting is 1.
15.7.8. Creat ing devices on host nodes
The vi rsh no d ed ev-create file command allows you to create a device on a host node and then assign it to a guest virtual machine. lib virt normally detects which host nodes are available for use automatically, but this command allows for the registration of host hardware that lib virt did not detect. The file should contain the XML for the top level <d evi ce> description of the node device.
To stop this device, use the no d ed ev-d estro y device command.
179
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
15.7.9. Det aching a node device
The vi rsh no d ed ev-d etach detaches the nodedev from the host so it can be safely used by guests via <ho std ev> passthrough. This action can be reversed with the no d ed ev-reattach command but it is done automatically for managed services. This command also accepts no d ed ev-
d ettach.
Note that different drivers expect the device to be bound to different dummy devices. Using the --
d ri ver argument allows you to specify the desired backend driver.
15.7.10. Dump a Device
The vi rsh no d ed ev-d umpxml device dumps the <d evi ce> XML representation for the given node device, including information such as the device name, which BUS owns the device, the vendor, and product ID as well as any capabilities of the device as is usable by libvirt, where it may specify what is supported. The argument device can either be a device name or WWN pair in WWNN,
WWPN format (HBA only).
15.7.11. List devices on a node
The vi rsh no d ed ev-l i st cap --tree command lists all the devices available on the node that are known by lib virt . cap is used to filter the list by capability types, each separated by a comma and cannot be used with --tree. Using the argument --tree, puts the output into a tree structure as shown:
# virsh nodedev-list --tree
computer
|
+- net_lo_00_00_00_00_00_00
+- net_macvtap0_52_54_00_12_fe_50
+- net_tun0
+- net_virbr0_nic_52_54_00_03_7d_cb
+- pci_0000_00_00_0
+- pci_0000_00_02_0
+- pci_0000_00_16_0
+- pci_0000_00_19_0
| |
| +- net_eth0_f0_de_f1_3a_35_4f
(this is a partial screen)
15.7.12. T riggering a reset for a node
The no d ed ev-reset nodedev command triggers a device reset for the specified nodedev. Running this command is useful prior to transferring a node device between guest virtual machine pass through or the host physical machine. libvirt will do this action implicitly when required, but this command allows an explicit reset when needed.
15.8. St art ing, suspending, resuming, saving and rest oring a guest virt ual machine
15.8.1. St art ing a defined domain
180
Chapt er 1 5. Managing guest virt ual machines wit h virsh
The vi rsh start domain --co nso l e --paused --auto d estro y --bypass-cache --
fo rce-bo o t --pass-fd s command starts a inactive domain that was already defined but whose state is inactive since its last managed save state or a fresh boot. The command can take the following arguments:
--co nso l e - will boot the domain attaching to the console
--paused - If this is supported by the driver it will boot the domain and then put it into a paused state
--auto d estro y - the guest virtual machine is automatically destroyed when the virsh session closes or the connection to libvirt closes, or it otherwise exits
--bypass-cache - used if the domain is in the managedsave state. If this is used, it will restore the guest virtual machine, avoiding the system cache. Note this will slow down the restore process.
--fo rce-bo o t - discards any managedsave options and causes a fresh boot to occur
--pass-fd s - is a list of additional arguments separated by commas, which are passed onto the guest virtual machine.
15.8.2. Suspending a guest virt ual machine
Suspend a guest virtual machine with vi rsh:
# virsh suspend {domain-id, domain-name or domain-uuid}
When a guest virtual machine is in a suspended state, it consumes system RAM but not processor resources. Disk and network I/O does not occur while the guest virtual machine is suspended. This operation is immediate and the guest virtual machine can be restarted with the resume
(
Section 15.8.6, “ Resuming a guest virtual machine” ) option.
15.8.3. Suspending a running domain
The vi rsh d o mpmsuspend domain --d urati o n --targ et command will take a running domain and suspended it so it can be placed into one of three possible states (S3, S4, or a hybrid of the two).
# vi rsh d o mpmsuspend rhel7 --d urati o n 10 0 --targ et mem
This command can take the following arguments:
--d urati o n - sets the duration for the state change in seconds
--targ et - can be either mem (suspend to R AM (S3))d i sk (suspend to d i sk (S4 )), or hybri d (hybri d suspend )
15.8.4 . Waking up a domain from pmsuspend st at e
This command will inject a wake-up alert to a guest that is in a pmsuspend state, rather than waiting for the duration time set to expire. This operation will not fail if the domain is running.
# d o mpmwakeup rhel7
181
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
This command requires the name of the domain, rhel7 for example as shown.
15.8.5. Undefining a domain
This command will undefine a domain. Although it can work on a running domain it will convert the running domain into a transient domain without stopping it. If the domain is inactive, the domain configuration is removed.
The vi rsh und efi nedomain--manag ed -save--snapsho ts-metad ata --sto rag e --
remo ve-al l -sto rag e --wi pe-sto rag e command can take the following arguments:
--manag ed -save - this argument guarantees that any managed save image is also cleaned up.
Without using this argument, attempts to undefine a domain with a managed save will fail.
--snapsho ts-metad ata - this argument guarantees that any snapshots (as shown with
snapho t-l i st) are also cleaned up when undefining an inactive domain. Note that any attempts to undefine an inactive domain with snapshot metadata will fail. If this argument is used and the domain is active, it is ignored.
--sto rag e - using this argument requires a comma separated list of volume target names or source paths of storage volumes to be removed along with the undefined domain. This action will undefine the storage volume before it is removed. Note that this can only be done with inactive domains. Note too that this will only work with storage volumes that are managed by libvirt.
--remo ve-al l -sto rag e - in addition to undefining the domain, all associated storage volumes are deleted.
--wi pe-sto rag e - in addition to deleting the storage volume, the contents are wiped.
15.8.6. Resuming a guest virt ual machine
Restore a suspended guest virtual machine with vi rsh using the resume option:
# virsh resume {domain-id, domain-name or domain-uuid}
This operation is immediate and the guest virtual machine parameters are preserved for suspend and resume operations.
15.8.7. Save a guest virt ual machine
Save the current state of a guest virtual machine to a file using the vi rsh command:
# vi rsh save {domain-name|domain-id|domain-uuid} state-file --bypass-
cache --xml --runni ng --paused --verbo se
This stops the guest virtual machine you specify and saves the data to a file, which may take some time given the amount of memory in use by your guest virtual machine. You can restore the state of the guest virtual machine with the resto re (
Section 15.8.11, “ Restore a guest virtual machine” )
option. Save is similar to pause, instead of just pausing a guest virtual machine the present state of the guest virtual machine is saved.
The vi rsh save command can take the following arguments:
--bypass-cache - causes the restore to avoid the file system cache but note that using this flag may slow down the restore operation.
182
Chapt er 1 5. Managing guest virt ual machines wit h virsh
--xml - this argument must be used with an XML file name. Although this argument is usually omitted, it can be used to supply an alternative XML file for use on a restored guest virtual machine with changes only in the host-specific portions of the domain XML. For example, it can be used to account for the file naming differences in underlying storage due to disk snapshots taken after the guest was saved.
--runni ng - overrides the state recorded in the save image to start the domain as running.
--paused - overrides the state recorded in the save image to start the domain as paused.
--verbo se - displays the progress of the save.
If you want to restore the guest virtual machine directly from the XML file, the vi rsh resto re command will do just that. You can monitor the process with the d o mjo bi nfo and cancel it with the
d o mjo babo rt.
15.8.8. Updat ing t he domain XML file t hat will be used for rest oring t he guest
The vi rsh save-i mag e-d efi ne file xml --runni ng | --paused command will update the domain XML file that will be used when the specified file is later used during the vi rsh resto re command. The xml argument must be an XML file name containing the alternative XML with changes only in the host physical machine specific portions of the domain XML. For example, it can be used to account for the file naming differences resulting from creating disk snapshots of underlying storage after the guest was saved. The save image records if the domain should be restored to a running or paused state. Using the arguments --runni ng or --paused dictates the state that is to be used.
15.8.9. Ext ract ing t he domain XML file
save-i mag e-d umpxml file --securi ty-i nfo command will extract the domain XML file that was in effect at the time the saved state file (used in the vi rsh save command) was referenced.
Using the --securi ty-i nfo argument includes security sensitive information in the file.
15.8.10. Edit Domain XML configurat ion files
save-i mag e-ed i t file --runni ng --paused command edits the XML configuration file that is associated with a saved file that was created by the vi rsh save command.
Note that the save image records whether the domain should be restored to a --runni ng or --
paused state. Without using these arguments the state is determined by the file itself. By selecting --
runni ng or --paused you can overwrite the state that vi rsh resto re should use.
15.8.11. Rest ore a guest virt ual machine
Restore a guest virtual machine previously saved with the vi rsh save command ( Section 15.8.7,
“ Save a guest virtual machine” ) using vi rsh:
# virsh restore state-file
This restarts the saved guest virtual machine, which may take some time. The guest virtual machine's name and UUID are preserved but are allocated for a new id.
The vi rsh resto re state-file command can take the following arguments:
--bypass-cache - causes the restore to avoid the file system cache but note that using this flag may slow down the restore operation.
183
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
--xml - this argument must be used with an XML file name. Although this argument is usually omitted, it can be used to supply an alternative XML file for use on a restored guest virtual machine with changes only in the host-specific portions of the domain XML. For example, it can be used to account for the file naming differences in underlying storage due to disk snapshots taken after the guest was saved.
--runni ng - overrides the state recorded in the save image to start the domain as running.
--paused - overrides the state recorded in the save image to start the domain as paused.
15.9. Shut t ing down, reboot ing and force-shut down of a guest virt ual machine
15.9.1. Shut down a guest virt ual machine
Shut down a guest virtual machine using the vi rsh shutd o wn command:
# virsh shutdown {domain-id, domain-name or domain-uuid} [--mo d e method]
You can control the behavior of the rebooting guest virtual machine by modifying the o n_shutd o wn parameter in the guest virtual machine's configuration file.
15.9.2. Shut t ing down Red Hat Ent erprise Linux 6 guest s on a Red Hat
Ent erprise Linux 7 host
Installing Red Hat Enterprise Linux 6 guest virtual machines with the Mi ni mal i nstal l ati o n option does not install the acpid package. Red Hat Enterprise Linux 7 no longer requires this package, as it has been taken over by systemd . However, Red Hat Enterprise Linux 6 guest virtual machines running on a Red Hat Enterprise Linux 7 host still require it.
Without the acpid package, the Red Hat Enterprise Linux 6 guest virtual machine does not shut down when the vi rsh shutd o wn command is executed. The vi rsh shutd o wn command is designed to gracefully shut down guest virtual machines.
Using vi rsh shutd o wn is easier and safer for system administration. Without graceful shut down with the vi rsh shutd o wn command a system administrator must log into a guest virtual machine manually or send the C trl -Al t-D el key combination to each guest virtual machine.
Note
Other virtualized operating systems may be affected by this issue. The vi rsh shutd o wn command requires that the guest virtual machine operating system is configured to handle
ACPI shut down requests. Many operating systems require additional configuration on the guest virtual machine operating system to accept ACPI shut down requests.
Pro ced u re 15.4 . Wo rkaro u n d f o r Red Hat En t erp rise Lin u x 6 g u est s
1. In st all t h e acp id p ackag e
The acpi d service listen and processes ACPI requests.
Log into the guest virtual machine and install the acpid package on the guest virtual machine:
184
Chapt er 1 5. Managing guest virt ual machines wit h virsh
# yum install acpid
2. En ab le t h e acp id service
Set the acpi d service to start during the guest virtual machine boot sequence and start the service:
# systemctl enable acpid
# service acpid start
3. Prep are g u est d o main xml
Edit the domain XML file to include the following element. Replace the virtio serial port with
o rg . q emu. g uest_ag ent. 0 and use your guest's name instead of $guestname
<channel type= 'unix' >
<source mode= 'bind' path= '/var/lib/libvirt/qemu/{$guestname}.agent' />
<target type= 'virtio' name= 'org.qemu.guest_agent.0' />
</channel>
Fig u re 15.2. G u est XML rep lacemen t
4. In st all t h e Q EMU g u est ag en t
Install the QEMU guest agent (QEMU-GA) and start the service as directed in Chapter 11,
. If you are running a Windows guest there are instructions in
this chapter for that as well.
5. Sh u t d o wn t h e g u est a. Run the following commands
# vi rsh l i st --al l - this command lists all of the known domains
Id Name State
----------------------------------
rhel6 running b. Shut down the guest virtual machine
# vi rsh shutd o wn rhel6
Domain rhel6 is being shutdown c. Wait a few seconds for the guest virtual machine to shut down.
185
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# vi rsh l i st --al l
Id Name State
----------------------------------
. rhel6 shut off d. Start the domain named rhel7, with the XML file you edited.
# vi rsh start rhel 6 e. Shut down the acpi in the rhel7 guest virtual machine.
# vi rsh shutd o wn --mo d e acpi rhel7 f. List all the domains again, rhel6 should still be on the list, and it should indicate it is shut off.
# vi rsh l i st --al l
Id Name State
----------------------------------
rhel6 shut off g. Start the domain named rhel7, with the XML file you edited.
# vi rsh start rhel6 h. Shut down the rhel6 guest virtual machine guest agent.
# vi rsh shutd o wn --mo d e ag ent rhel6 i. List the domains. rhel7 should still be on the list, and it should indicate it is shut off
# vi rsh l i st --al l
Id Name State
----------------------------------
rhel6 shut off
The guest virtual machine will shut down using the vi rsh shutd o wn command for the consecutive shutdowns, without using the workaround described above.
In addition to the method described above, a guest can be automatically shutdown, by stopping the
libvirt-guest service. Refer to
Section 15.9.3, “ Manipulating the libvirt-guests configuration settings”
for more information on this method.
15.9.3. Manipulat ing t he libvirt -guest s configurat ion set t ings
The libvirt-guests service has parameter settings that can be configured to assure that the guest is shutdown properly. It is a package that is a part of the libvirt installation and is installed by default.
This service automatically saves guests to the disk when the host shuts down, and restores them to their pre-shutdown state when the host reboots. By default, this setting is set to suspend the guest. If you want the guest to be shutoff, you will need to change one of the parameters of the libvirt-guests configuration file.
Pro ced u re 15.5. Ch an g in g t h e lib virt - g u est s service p aramet ers t o allo w f o r t h e g racef u l sh u t d o wn o f g u est s
186
Chapt er 1 5. Managing guest virt ual machines wit h virsh g racef u l sh u t d o wn o f g u est s
The procedure described here allows for the graceful shutdown of guest virtual machines when the host physical machine is stuck, powered off, or needs to be restarted.
1. O p en t h e co n f ig u rat io n f ile
The configuration file is located in /etc/sysco nfi g /l i bvi rt-g uests. Edit the file, remove the comment mark (#) and change the ON_SHUTDOWN=suspend to
ON_SHUTDOWN=shutdown. Remember to save the change.
$ vi /etc/sysconfig/libvirt-guests
# URIs to check for running guests
# example: URIS='default xen:/// vbox+tcp://host/system lxc:///'
#URIS=default
# action taken on host boot
# - start all guests which were running on shutdown are started on boot
# regardless on their autostart settings
1
# - ignore libvirt-guests init script won't start any guest on boot, however,
2
# guests marked as autostart will still be automatically started by
3
# libvirtd
4
#ON_BOOT=start
5
6
# Number of seconds to wait between each guest start. Set to 0 to allow
7
# parallel startup.
#START_DELAY=0
# action taken on host shutdown
# - suspend all running guests are suspended using virsh managedsave
# - shutdown all running guests are asked to shutdown. Please be careful with
# this settings since there is no way to distinguish between a
# guest which is stuck or ignores shutdown requests and a guest
# which just needs a long time to shutdown. When setting
# ON_SHUTDOWN=shutdown, you must also set
187
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
SHUTDOWN_TIMEOUT to a
# value suitable for your guests.
ON_SHUTDOWN=shutdown
# If set to non-zero, shutdown will suspend guests concurrently.
Number of
# guests on shutdown at any time will not exceed number set in this variable.
#PARALLEL_SHUTDOWN=0
# Number of seconds we're willing to wait for a guest to shut down.
If parallel
# shutdown is enabled, this timeout applies as a timeout for shutting down all
# guests on a single URI defined in the variable URIS. If this is
0, then there
# is no time out (use with caution, as guests might not respond to a shutdown
# request). The default value is 300 seconds (5 minutes).
#SHUTDOWN_TIMEOUT=300
# If non-zero, try to bypass the file system cache when saving and
# restoring guests, even though this may give slower operation for
# some file systems.
#BYPASS_CACHE=0
URIS - checks the specified connections for a running guest. The Default setting functions in the same manner as virsh does when no explicit URI is set In addition, one can explicitly set the URI from /etc/l i bvi rt/l i bvi rt. co nf. It should be noted that when using the libvirt configuration file default setting, no probing will be used.
ON_BOOT - specifies the action to be done to / on the guests when the host boots. The
start option starts all guests that were running prior to shutdown regardless on their autostart settings. The ignore option will not start the formally running guest on boot, however, any guest marked as autostart will still be automatically started by libvirtd.
The START_DELAY - sets a delay interval in between starting up the guests. This time period is set in seconds. Use the 0 time setting to make sure there is no delay and that all guests are started simultaneously.
ON_SHUTDOWN - specifies the action taken when a host shuts down. Options that can be set include: suspend which suspends all running guests using vi rsh
manag ed save and shutdown which shuts down all running guests. It is best to be careful with using the shutd o wn option as there is no way to distinguish between a guest which is stuck or ignores shutdown requests and a guest that just needs a longer time to shutdown. When setting the ON_SHUTDOWN=shutdown, you must also set SHUTDOWN_TIMEOUT to a value suitable for the guests.
PARALLEL_SHUTDOWN Dictates that the number of guests on shutdown at any time will not exceed number set in this variable and the guests will be suspended concurrently.
If set to 0 , then guests are not shutdown concurrently.
Number of seconds to wait for a guest to shut down. If SHUTDOWN_TIMEOUT is enabled, this timeout applies as a timeout for shutting down all guests on a single URI defined in the variable URIS. If SHUTDOWN_TIMEOUT is set to 0, then there is no time out (use with caution, as guests might not respond to a shutdown request). The default value is 300 seconds (5 minutes).
BYPASS_CACHE can have 2 values, 0 to disable and 1 to enable. If enabled it will bypass the file system cache when guests are restored. Note that setting this may effect performance and may cause slower operation for some file systems.
188
Chapt er 1 5. Managing guest virt ual machines wit h virsh
2. St art lib virt - g u est s service
If you have not started the service, start the libvirt-guests service. Do not restart the service as this will cause all running domains to shutdown.
15.9.4 . Reboot ing a guest virt ual machine
Reboot a guest virtual machine using vi rsh rebo o t command. Remember that this action will return once it has executed the reboot, but there may be a time lapse from that point until the domain actually reboots.
#virsh reboot {domain-id, domain-name or domain-uuid} [--mo d e method]
You can control the behavior of the rebooting guest virtual machine by modifying the o n_rebo o t element in the guest virtual machine's configuration file.
By default, the hypervisor will try to pick a suitable shutdown method. To specify an alternative method, the --mo d e argument can specify a comma separated list which includes i ni tctl , acpi ,
ag ent, si g nal . The order in which drivers will try each mode is undefined, and not related to the order specified in virsh. For strict control over ordering, use a single mode at a time and repeat the command.
15.9.5. Forcing a guest virt ual machine t o st op
Force a guest virtual machine to stop with the vi rsh d estro y command:
# virsh destroy {domain-id, domain-name or domain-uuid} [--g raceful ]
This command does an immediate ungraceful shutdown and stops the specified guest virtual machine. Using vi rsh d estro y can corrupt guest virtual machine file systems. Use the d estro y option only when the guest virtual machine is unresponsive. If you want to initiate a graceful shutdown, use the vi rsh d estro y --g raceful command.
15.9.6. Reset t ing a virt ual machine
vi rsh reset domain resets the domain immediately without any guest shutdown. A reset emulates the power reset button on a machine, where all guest hardware sees the RST line and re-initializes the internal state. Note that without any guest virtual machine OS shutdown, there are risks for data loss.
15.10. Ret rieving guest virt ual machine informat ion
15.10.1. Get t ing t he domain ID of a guest virt ual machine
To get the domain ID of a guest virtual machine:
# virsh domid {domain-name or domain-uuid}
15.10.2. Get t ing t he domain name of a guest virt ual machine
To get the domain name of a guest virtual machine:
189
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# virsh domname {domain-id or domain-uuid}
15.10.3. Get t ing t he UUID of a guest virt ual machine
To get the Universally Unique Identifier (UUID) for a guest virtual machine:
# virsh domuuid {domain-id or domain-name}
An example of vi rsh d o muui d output:
# virsh domuuid r5b2-mySQL01
4a4c59a7-ee3f-c781-96e4-288f2862f011
15.10.4 . Displaying guest virt ual machine informat ion
Using vi rsh with the guest virtual machine's domain ID, domain name or UUID you can display information on the specified guest virtual machine:
# virsh dominfo {domain-id, domain-name or domain-uuid}
This is an example of vi rsh d o mi nfo output:
# virsh dominfo vr-rhel6u1-x86_64-kvm
Id: 9
Name: vr-rhel6u1-x86_64-kvm
UUID: a03093a1-5da6-a2a2-3baf-a845db2f10b9
OS Type: hvm
State: running
CPU(s): 1
CPU time: 21.6s
Max memory: 2097152 kB
Used memory: 1025000 kB
Persistent: yes
Autostart: disable
Security model: selinux
Security DOI: 0
Security label: system_u:system_r:svirt_t:s0:c612,c921 (permissive)
15.11. St orage pool commands
The following commands manipulate storage pools. Using libvirt you can manage various storage solutions, including files, raw partitions, and domain-specific formats, used to provide the storage volumes visible as devices within virtual machines. For more detailed information about this feature, see more information at libvirt.org
. Many of the commands for storage pools are similar to the ones used for domains.
15.11.1. Searching for a st orage pool XML
The fi nd -sto rag e-po o l -so urces type srcSpec command displays the XML describing all storage pools of a given type that could be found. If srcSpec is provided, it is a file that contains XML to further restrict the query for pools.
190
Chapt er 1 5. Managing guest virt ual machines wit h virsh
The fi nd -sto rag e-po o l -so urces-as type host port initiator displays the XML describing all storage pools of a given type that could be found. If host, port, or initiator are provided, they control where the query is performed.
The po o l -i nfo pool-or-uuid command will list the basic information about the specified storage pool object. This command requires the name or UUID of the storage pool. To retrieve this information, use the po o l -l i st
The po o l -l i st --i nacti ve --al l --persi stent --transi ent --auto start --no -
auto start --d etai l s<type> command lists all storage pool objects known to libvirt. By default, only active pools are listed; but using the --i nacti ve argument lists just the inactive pools, and using the --al l argument lists all of the storage pools.
In addition to those arguments there are several sets of filtering flags that can be used to filter the content of the list. --persi stent restricts the list to persistent pools, --transi ent restricts the list to transient pools, --auto start restricts the list to autostarting pools and finally --no -auto start restricts the list to the storage pools that have autostarting disabled.
For all storage pool commands which require a type, the pool types must be separated by comma.
The valid pool types include: d i r, fs, netfs, l o g i cal , d i sk, i scsi , scsi , mpath, rbd , and
sheepd o g .
The --d etai l s option instructs virsh to additionally display pool persistence and capacity related information where available.
Note
When this command is used with older servers, it is forced to use a series of API calls with an inherent race, where a pool might not be listed or might appear more than once if it changed its state between calls while the list was being collected. Newer servers however, do not have this problem.
The po o l -refresh pool-or-uuid refreshes the list of volumes contained in pool.
15.11.2. Creat ing, defining, and st art ing st orage pools
1 5 .1 1 .2 .1 . Building a st o rage po o l
The po o l -bui l d pool-or-uuid --o verwri te --no -o verwri te command builds a pool with a specified pool name or UUID. The arguments --o verwri te and --no -o verwri te can only be used for a pool whose type is file system. If neither argument is specified, and the pool is a file system type pool, then the resulting build will only make the directory.
If --no -o verwri te is specified, it probes to determine if a file system already exists on the target device, returning an error if it exists, or using mkfs to format the target device if it does not. If --
o verwri te is specified, then the mkfs command is executed and any existing data on the target device is overwritten.
1 5 .1 1 .2 .2 . Cre at ing and de fining a st o rage po o l fro m an XML file
The po o l -create file creates and starts a storage pool its associated XML file.
The po o l -d efi ne file creates, but does not start, a storage pool object from the XML file.
191
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
1 5 .1 1 .2 .3. Cre at ing and st art ing a st o rage po o l fro m raw param e t e rs
The po o l -create-as name --pri nt-xml type so urce-ho st so urce-path so urce-d ev
so urce-name <targ et> --so urce-fo rmat <fo rmat> command creates and starts a pool object name from the raw parameters given.
If --pri nt-xml is specified, then it prints the XML of the storage pool object without creating the pool. Otherwise, the pool requires a type in order to be built. For all storage pool commands which require a type, the pool types must be separated by comma. The valid pool types include: d i r, fs,
netfs, l o g i cal , d i sk, i scsi , scsi , mpath, rbd , and sheepd o g .
The po o l -d efi ne-as name --pri nt-xml type so urce-ho st so urce-path so urce-d ev
so urce-name <targ et> --so urce-fo rmat <fo rmat> command creates, but does not start, a pool object name from the raw parameters given.
If --pri nt-xml is specified, then it prints the XML of the pool object without defining the pool.
Otherwise, the pool has to have a specified type. For all storage pool commands which require a
type, the pool types must be separated by comma. The valid pool types include: d i r, fs, netfs,
l o g i cal , d i sk, i scsi , scsi , mpath, rbd , and sheepd o g .
The po o l -start pool-or-uuid starts the specified storage pool, which was previously defined but inactive.
1 5 .1 1 .2 .4 . Aut o -st art ing a st o rage po o l
The po o l -auto start pool-or-uuid --d i sabl e command enables or disables a storage pool to automatically start at boot. This command requires the pool name or UUID. To disable the po o l -
auto start command use the --d i sabl e argument.
15.11.3. St opping and delet ing st orage pools
The po o l -d estro y pool-or-uuid stops a storage pool. Once stopped, libvirt will no longer manage the pool but the raw data contained in the pool is not changed, and can be later recovered with the po o l -create command.
The po o l -d el ete pool-or-uuid destroys the resources used by the specified storage pool. It is important to note that this operation is non-recoverable and non-reversible. However, the pool structure will still exist after this command, ready to accept the creation of new storage volumes.
The po o l -und efi ne pool-or-uuid command undefines the configuration for an inactive pool.
15.11.4 . Creat ing an XML dump file for a pool
The po o l -d umpxml --i nacti ve pool-or-uuid command returns the XML information about the specified storage pool object. Using --i nacti ve dumps the configuration that will be used on next start of the pool as opposed to the current pool configuration.
15.11.5. Edit ing t he st orage pool's configurat ion file
The po o l -ed i t pool-or-uuid opens the specified storage pool's XML configuration file for editing.
This method is the only method that should be used to edit an XML configuration file as it does error checking before applying.
15.11.6. Convert ing st orage pools
192
Chapt er 1 5. Managing guest virt ual machines wit h virsh
The po o l -name uuid command converts the specified UUID to a pool name.
The po o l -uui d pool command returns the UUID of the specified pool.
15.12. St orage Volume Commands
This section covers all commands for creating, deleting, and managing storage volumes. It is best to do this once you have created a storage pool as the storage pool name or UUID will be required. For information on storage pools refer to
. For information on storage volumes
refer to, Chapter 14, Volumes .
15.12.1. Creat ing st orage volumes
The vo l -create-fro m pool-or-uuid file --i nputpo o l pool-or-uuid vol-name-or-
key-or-path creates a volume, using another volume as input. This command requires a pool-or-
uuid which is the name or UUID of the storage pool to create the volume in.
The file argument contains is the XML file and path containing the volume definition. The --
i nputpo o l pool-or-uuid argument specifies the name or uuid of the storage pool the source volume is in. The vol-name-or-key-or-path argument specifies the name or key or path of the source volume. For some examples, refer to
Section 14.1, “ Creating volumes” .
The vo l -create-as command creates a volume from a set of arguments. The pool-or-uuid argument contains the name or UUID of the storage pool to create the volume in.
vo l -create-as pool-or-uuid name capacity --al l o cati o n <si ze> --fo rmat
<stri ng > --backi ng -vo l <vo l -name-o r-key-o r-path> --backi ng -vo l -fo rmat
<stri ng >
name is the name of the new volume. capacity is the size of the volume to be created, as a scaled integer, defaulting to bytes if there is no suffix. --al l o cati o n <si ze> is the initial size to be allocated in the volume, also as a scaled integer defaulting to bytes. --fo rmat <stri ng > is used in file based storage pools to specify the volume file format which is a string of acceptable formats separated by a comma. Acceptable formats include raw, bo chs, q co w, q co w2, vmd k, --backi ng -
vo l vol-name-or-key-or-path is the source backing volume to be used if taking a snapshot of an existing volume. --backi ng -vo l -fo rmat string is the format of the snapshot backing volume which is a string of formats separated by a comma. Accepted values include: raw, bo chs,
q co w, q co w2, , vmd k, and ho st_d evi ce. These are, however, only meant for file based storage pools.
1 5 .1 2 .1 .1 . Cre at ing a st o rage vo lum e fro m an XML file
The vo l -create pool-or-uuid file creates a storage volume from an XML file. This command also requires the pool-or-uuid, which is the name or UUID of the storage pool to create the volume in.
The file argument contains the path with the volume definition's XML file. An easy way to create the
XML file is to use the vo l -d umpxml command to obtain the definition of a pre-existing volume.
virsh vol-dumpxml --pool storagepool1 appvolume1 > newvolume.xml
virsh edit newvolume.xml virsh vol-create differentstoragepool newvolume.xml
Other options available include: the --i nacti ve option to list inactive guest virtual machines (that is, guest virtual machines that have been defined but are not currently active), and
193
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
the --al l option lists all guest virtual machines. For example:
1 5 .1 2 .1 .2 . Clo ning a st o rage vo lum e
The vo l -cl o ne --po o l pool-or-uuid vol-name-or-key-or-path name command clones an existing storage volume. Although the vo l -create-fro m may also be used, it is not the recommended way to clone a storage volume. The --po o l pool-or-uuid argument is the name or UUID of the storage pool to create the volume in. The vol-name-or-key-or-path argument is the name or key or path of the source volume. Using a name argument refers to the name of the new volume.
15.12.2. Delet ing st orage volumes
The vo l -d el ete --po o l pool-or-uuid vol-name-or-key-or-path command deletes a given volume. The command requires a specific --po o l pool-or-uuid which is the name or UUID of the storage pool the volume is in. The option vol-name-or-key-or-path is the name or key or path of the volume to delete.
The vo l -wi pe --po o l pool-or-uuid --al g o ri thm algorithm vol-name-or-key-or-
pathcommand wipes a volume, to ensure data previously on the volume is not accessible to future reads. The command requires a --po o l pool-or-uuid which is the name or UUID of the storage pool the volume is in. The vol-name-or-key-or-path contains the name or key or path of the volume to wipe. Note that it is possible to choose different wiping algorithms instead of re-writing volume with zeroes, via the argument --al g o ri thm and using one of the following supported algorithm types:
Note
The availability of algorithms may be limited by the version of the "scrub" binary installed on the host.
zero - 1-pass all zeroes
nnsa - 4-pass NNSA Policy Letter NAP-14.1-C (XVI-8) for sanitizing removable and non-removable hard disks: random x2, 0x00, verify.
d o d - 4-pass DoD 5220.22-M section 8-306 procedure for sanitizing removeable and nonremoveable rigid disks: random, 0x00, 0xff, verify.
bsi - 9-pass method recommended by the German Center of Security in Information
Technologies (http://www.bsi.bund.de): 0xff, 0xfe, 0xfd, 0xfb, 0xf7, 0xef, 0xdf, 0xbf, 0x7f.
g utmann - The canonical 35-pass sequence described in Gutmann’s paper.
schnei er - 7-pass method described by Bruce Schneier in "Applied Cryptography" (1996):
0x00, 0xff, random x5.
pfi tzner7 - Roy Pfitzner’s 7-random-pass method: random x7
pfi tzner33 - Roy Pfitzner’s 33-random-pass method: random x33.
rand o m - 1-pass pattern: random.
15.12.3. Dumping st orage volume informat ion t o an XML file
vo l -d umpxml --po o l pool-or-uuid vol-name-or-key-or-path command takes the volume information as an XML dump to a specified file.
194
Chapt er 1 5. Managing guest virt ual machines wit h virsh
This command requires a --po o l pool-or-uuid, which is the name or UUID of the storage pool the volume is in. vol-name-or-key-or-path is the name or key or path of the volume to place the resulting XML file.
15.12.4 . List ing volume informat ion
The vo l -i nfo --po o l pool-or-uuid vol-name-or-key-or-path command lists basic information about the given storage volume --po o l , where pool-or-uuid is the name or UUID of the storage pool the volume is in. vol-name-or-key-or-path is the name or key or path of the volume to return information for.
The vo l -l i st--po o l pool-or-uuid --d etai l s lists all of volumes in the specified storage pool. This command requires --po o l pool-or-uuid which is the name or UUID of the storage pool. The --d etai l s option instructs virsh to additionally display volume type and capacity related information where available.
15.12.5. Ret rieving st orage volume informat ion
The vo l -po o l --uui d vol-key-or-path command returns the pool name or UUID for a given volume. By default, the pool name is returned. If the --uui d option is given, the pool UUID is returned instead. The command requires the vol-key-or-path which is the key or path of the volume for which to return the requested information.
The vo l -path --po o l pool-or-uuid vol-name-or-key command returns the path for a given volume. The command requires --po o l pool-or-uuid, which is the name or UUID of the storage pool the volume is in. It also requires vol-name-or-key which is the name or key of the volume for which the path has been requested.
The vo l -name vol-key-or-path command returns the name for a given volume, where vol-key-
or-path is the key or path of the volume to return the name for.
The vo l -key --po o l pool-or-uuid vol-name-or-path command returns the volume key for a given volume where --po o l pool-or-uuid is the name or UUID of the storage pool the volume is in and vol-name-or-path is the name or path of the volume to return the volume key for.
15.12.6. Uploading and downloading st orage volumes
This section will instruct how to upload and download information to and from storage volumes.
1 5 .1 2 .6 .1 . Uplo ading co nt e nt s t o a st o rage vo lum e
The vo l -upl o ad --po o l pool-or-uuid --o ffset bytes --l eng th bytes vol-name-
or-key-or-path local-file command uploads the contents of specified local-file to a storage volume. The command requires --po o l pool-or-uuid which is the name or UUID of the storage pool the volume is in. It also requires vol-name-or-key-or-path which is the name or key or path of the volume to wipe. The --o ffset argument is the position in the storage volume at which to start writing the data. --l eng th length dictates an upper limit for the amount of data to be uploaded.
An error will occur if the local-file is greater than the specified --l eng th.
1 5 .1 2 .6 .2 . Do wnlo ading t he co nt e nt s fro m a st o rage vo lum e
The vo l -d o wnl o ad --po o l pool-or-uuid --o ffset bytes -l eng th bytes vol-name-
or-key-or-path local-file command downloads the contents of local-file from a storage volume.
The command requires a --po o l pool-or-uuid which is the name or UUID of the storage pool
195
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
that the volume is in. It also requires vol-name-or-key-or-path which is the name or key or path of the volume to wipe. Using the argument --o ffset dictates the position in the storage volume at which to start reading the data. --l eng th length dictates an upper limit for the amount of data to be downloaded.
15.12.7. Re-siz ing st orage volumes
The vo l -resi ze --po o l pool-or-uuid vol-name-or-path pool-or-uuid capacity -
-al l o cate --d el ta --shri nk command re-sizes the capacity of the given volume, in bytes.The
command requires --po o l pool-or-uuid which is the name or UUID of the storage pool the volume is in. This command also requires vol-name-or-key-or-path is the name or key or path of the volume to re-size.
The new capacity might be sparse unless --al l o cate is specified. Normally, capacity is the new size, but if --d el ta is present, then it is added to the existing size. Attempts to shrink the volume will fail unless --shri nk is present.
Note that capacity cannot be negative unless --shri nk is provided and a negative sign is not necessary. capacity is a scaled integer which defaults to bytes if there is no suffix. Note too that this
command is only safe for storage volumes not in use by an active guest. Refer to Section 15.5.17,
“ Using blockresize to change the size of a domain path” for live re-sizing.
15.13. Displaying per-guest virt ual machine informat ion
15.13.1. Displaying t he guest virt ual machines
To display the guest virtual machine list and their current states with vi rsh:
# virsh list
Other options available include:
--i nacti ve option lists the inactive guest virtual machines (that is, guest virtual machines that have been defined but are not currently active)
--al l option lists all guest virtual machines. For example:
# virsh list --all
Id Name State
----------------------------------
0 Domain-0 running
1 Domain202 paused
2 Domain010 inactive
3 Domain9600 crashed
There are seven states that can be visible using this command:
Running - The runni ng state refers to guest virtual machines which are currently active on a
CPU.
Idle - The i d l e state indicates that the domain is idle, and may not be running or able to run.
This can be caused because the domain is waiting on IO (a traditional wait state) or has gone to sleep because there was nothing else for it to do.
196
Chapt er 1 5. Managing guest virt ual machines wit h virsh
Paused - The paused state lists domains that are paused. This occurs if an administrator uses the paused button in vi rt-manag er or vi rsh suspend . When a guest virtual machine is paused it consumes memory and other resources but it is ineligible for scheduling and CPU resources from the hypervisor.
Shutdown - The shutd o wn state is for guest virtual machines in the process of shutting down.
The guest virtual machine is sent a shutdown signal and should be in the process of stopping its operations gracefully. This may not work with all guest virtual machine operating systems; some operating systems do not respond to these signals.
Shut off - The shut o ff state indicates that the domain is not running. This can be caused when a domain completely shuts down or has not been started.
Crashed - The crashed state indicates that the domain has crashed and can only occur if the guest virtual machine has been configured not to restart on crash.
Dying - Domains in the d yi ng state are in is in process of dying, which is a state where the domain has not completely shut-down or crashed.
--manag ed -save Although this flag alone does not filter the domains, it will list the domains that have managed save state enabled. In order to actually list the domains seperately you will need to use the --i nacti ve flag as well.
--name is specified domain names are printed in a list. If --uui d is specified the donain's UUID is printed instead. Using the flag --tabl e specifies that a table style output should be used. All three commands are mutually exclusive
--ti tl e This command must be used with --tabl e output. --ti tl ewill cause an extra column to be created in the table with the short domain description (title).
--persi stentincludes persistent domains in a list. Use the --transi ent argument.
--wi th-manag ed -save lists the domains that have been configured with managed save. To list the commands without it, use the command --wi tho ut-manag ed -save
--state-runni ng filters out for the domains that are running, --state-paused for paused domains, --state-shuto ff for domains that are turned off, and --state-o ther lists all states as a fallback.
--auto start this argument will cause the auto-starting domains to be listed. To list domains with this feature disabled, use the argument --no -auto start.
--wi th-snapsho t will list the domains whose snapshot images can be listed. To filter for the domains without a snapshot, use the argument --wi tho ut-snapsho t
$ virsh list --title --name
Id Name State
Title
0 Domain-0 running
Mailserver1
2 rhelvm paused
15.13.2. Displaying virt ual CPU informat ion
197
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
To display virtual CPU information from a guest virtual machine with vi rsh:
# virsh vcpuinfo {domain-id, domain-name or domain-uuid}
An example of vi rsh vcpui nfo output:
# virsh vcpuinfo rhel7
VCPU: 0
CPU: 2
State: running
CPU time: 7152.4s
CPU Affinity: yyyy
VCPU: 1
CPU: 2
State: running
CPU time: 10889.1s
CPU Affinity: yyyy
15.13.3. Configuring virt ual CPU affinit y
Examp le 15.3. Pin n in g vCPU t o a h o st p h ysical mach in e' s CPU
The vi rsh vcpupi n assigns a virtual CPU to a physical one.
# virsh vcpupin rhel7
VCPU: CPU Affinity
----------------------------------
0: 0-3
1: 0-3
The vcpupi n can take the following arguments:
--vcpu requires the vcpu number
[--cpul i st] >stri ng < lists the host physical machine's CPU number(s) to set, or omit an optional query
--co nfi g affects next boot
--l i ve affects the running domain
--current affects the current domain
15.13.4 . Displaying informat ion about t he virt ual CPU count s of a given domian
vi rsh vcpuco unt requires a domain name or a domain ID
# virsh vcpucount rhel6
198
Chapt er 1 5. Managing guest virt ual machines wit h virsh
maximum config 2 maximum live 2 current config 2 current live 2
The vcpuco unt can take the following arguments:
--maxi mum get maximum cap on vcpus
--acti ve get number of currently active vcpus
--l i ve get value from running domain
--co nfi g get value to be used on next boot
--current get value according to current domain state
--g uest count that is returned is from the perspective of the guest
15.13.5. Configuring virt ual CPU affinit y
To configure the affinity of virtual CPUs with physical CPUs:
# virsh vcpupin domain-id vcpu cpulist
The d o mai n-i d parameter is the guest virtual machine's ID number or name.
The vcpu parameter denotes the number of virtualized CPUs allocated to the guest virtual machine.The vcpu parameter must be provided.
The cpul i st parameter is a list of physical CPU identifier numbers separated by commas. The
cpul i st parameter determines which physical CPUs the VCPUs can run on.
Additional parameters such as --co nfi g effect the next boot, whereas --l i ve effects the running domain and --currenteffects the current domain
15.13.6. Configuring virt ual CPU count
By default, the virtual CPU (vCPU) count can only be changed on active guest domains. To change the settings for an inactive guest domain, use the --co nfi g flag. Modify the number of CPUs assigned to a guest virtual machine with the vi rsh command:
# vi rsh setvcpus {domain-name, domain-id or domain-uuid} count [[--
config] [--live] | [--current] [--guest]
The following parameters may be set for the vi rsh setvcpus command:
{domain-name, domain-id or domain-uuid} - Specifies the virtual machine.
--count - Specifies the number of virtual CPUs to set.
199
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Important
The --count value cannot exceed the number of CPUs that were assigned to the guest virtual machine when it was created.
--maximum - Sets a maximum virtual CPU limit on the next reboot.
--config - Configuration change takes effect on the next reboot.
--live - Configuration change takes effect on the running guest virtual machine.
--current - Configuration change takes effect on the current guest virtual machine.
--guest - Configuration change modifies the CPU state in the guest virtual machine.
Configurations set with --guest are reset when a guest is rebooted.
Note
For information on increasing vCPU performance by using multi-queue, refer to the Red Hat
Enterprise Linux Virtualization Tuning and Optimization Guide.
Examp le 15.4 . Ho t p lu g an d h o t - u n p lu g o f vCPU
To hotplug a vCPU, run the following command:
vi rsh setvcpus guestVM1 2 --live
In the above case, the number of vCPUs for guestVM1 is increased by two and this action will be performed while the guestVM1 is running, as indicated by the --live flag.
Likewise, to hot-unplug one vCPU from the same running guest, run the following:
vi rsh setvcpus guestVM1 1 --live
The count value may be limited by the host, hypervisor, or a limit coming from the original description of the guest domain. For Xen, you can only adjust the virtual CPUs of a running domain if the domain is para-virtualized.
If the --co nfi g flag is specified, the change is made to the stored XML configuration for the guest virtual machine domain, and will only take effect the next time the guest domain is started.
If --l i ve is specified, the guest virtual machine domain must be active, and the change takes place immediately. This flag will allow hotplugging of a vCPU. Both the --co nfi g and --l i ve flags may be specified together if supported by the hypervisor.
If --current is specified, the flag affects the current active guest virtual machine's state. When no flags are given, the --l i ve flag is used by default. Should this command run without a flag when there are no active guest virtual machines, it will fail. In some cases, the hypervisor will also assume the use of the --co nfi g . The hypervisor will also decide if the XML configuration is adjusted to make the change persistent.
200
Chapt er 1 5. Managing guest virt ual machines wit h virsh
The --maxi mum flag controls the maximum number of virtual CPUs that can be hotplugged the next time the domain is booted. As such, it must only be used with the --co nfi g flag, and not with the --l i ve flag.
15.13.7. Configuring memory allocat ion
To modify a guest virtual machine's memory allocation with vi rsh:
# virsh setmem {domain-id or domain-name} count
# vi rsh setmem vr-rhel6u1-x86_64-kvm --kilobytes 1025000
You must specify the co unt in kilobytes. The new count value cannot exceed the amount you specified when you created the guest virtual machine. Values lower than 64 MB are unlikely to work with most guest virtual machine operating systems. A higher maximum memory value does not affect active guest virtual machines. If the new value is lower than the available memory, it will shrink possibly causing the guest virtual machine to crash.
This command has the following flags:
[--domain] <string> domain name, id or uuid
[--size] <number> new memory size, as scaled integer (default KiB)
Valid memory units include:
b or bytes for bytes
KB for kilobytes (10
3
or blocks of 1,000 bytes)
k or KiB for kibibytes (2
10
or blocks of 1024 bytes)
MB for megabytes (10
6
or blocks of 1,000,000 bytes)
M or MiB for mebibytes (2
20
or blocks of 1,048,576 bytes)
GB for gigabytes (10
9
or blocks of 1,000,000,000 bytes)
G or GiB for gibibytes (2
30
or blocks of 1,073,741,824 bytes)
TB for terabytes (10
12
or blocks of 1,000,000,000,000 bytes)
T or TiB for tebibytes (2
40
or blocks of 1,099,511,627,776 bytes)
Note that all values will be rounded up to the nearest kibibyte by libvirt, and may be further rounded to the granularity supported by the hypervisor. Some hypervisors also enforce a minimum, such as 4000KiB (or 4000 x 2
10
or 4,096,000 bytes). The units for this value are determined by the optional attribute memory unit, which defaults to the kibibytes (KiB) as a unit of measure where the value given is multiplied by 2
10
or blocks of 1024 bytes.
--config takes affect next boot
--live controls the memory of the running domain
--current controls the memory on the current domain
201
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
15.13.8. Changing t he memory allocat ion for t he domain
The vi rsh setmaxmem domain size --co nfi g --l i ve --current allows the setting of the maximum memory allocation for a guest virtual machine as shown:
vi rsh setmaxmem rhel7 1024 --current
The size that can be given for the maximum memory is a scaled integer that by default is expressed in kibibytes, unless a supported suffix is provided. The following arguments can be used with this command:
--co nfi g - takes affect next boot
--l i ve - controls the memory of the running domain, providing the hypervisor supports this action as not all hypervisors allow live changes of the maximum memory limit.
--current - controls the memory on the current domain
15.13.9. Displaying guest virt ual machine block device informat ion
Use vi rsh d o mbl kstat to display block device statistics for a running guest virtual machine.
# virsh domblkstat GuestName block-device
15.13.10. Displaying guest virt ual machine net work device informat ion
Use vi rsh d o mi fstat to display network interface statistics for a running guest virtual machine.
# virsh domifstat GuestName interface-device
15.14 . Managing virt ual net works
This section covers managing virtual networks with the vi rsh command. To list virtual networks:
# virsh net-list
This command generates output similar to:
# virsh net-list
Name State Autostart
----------------------------------------default active yes vnet1 active yes vnet2 active yes
To view network information for a specific virtual network:
# virsh net-dumpxml NetworkName
This displays information about a specified virtual network in XML format:
# virsh net-dumpxml vnet1
202
Chapt er 1 5. Managing guest virt ual machines wit h virsh
<network>
<name>vnet1</name>
<uuid>98361b46-1581-acb7-1643-85a412626e70</uuid>
<forward dev='eth0'/>
<bridge name='vnet0' stp='on' forwardDelay='0' />
<ip address='192.168.100.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.100.128' end='192.168.100.254' />
</dhcp>
</ip>
</network>
Other vi rsh commands used in managing virtual networks are:
vi rsh net-auto start network-name — Autostart a network specified as network-name.
vi rsh net-create XMLfile — generates and starts a new network using an existing XML file.
vi rsh net-d efi ne XMLfile — generates a new network device from an existing XML file without starting it.
vi rsh net-d estro y network-name — destroy a network specified as network-name.
vi rsh net-name networkUUID — convert a specified networkUUID to a network name.
vi rsh net-uui d network-name — convert a specified network-name to a network UUID.
vi rsh net-start nameOfInactiveNetwork — starts an inactive network.
vi rsh net-und efi ne nameOfInactiveNetwork — removes the definition of an inactive network.
15.15. Migrat ing guest virt ual machines wit h virsh
Information on migration using virsh is located in the section entitled Live KVM Migration with virsh
Refer to
Section 5.4, “ Live KVM migration with virsh”
15.15.1. Int erface Commands
The following commands manipulate host interfaces and as such should not be run from the guest virtual machine. These commands should be run from a terminal on the host physical machine.
Warning
The commands in this section are only supported if the machine has the NetworkManager service disabled, and is using the network service instead.
Often, these host interfaces can then be used by name within domain <i nterface> elements (such as a system-created bridge interface), but there is no requirement that host interfaces be tied to any particular guest configuration XML at all. Many of the commands for host interfaces are similar to the ones used for domains, and the way to name an interface is either by its name or its MAC address.
203
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
However, using a MAC address for an i face argument only works when that address is unique (if an interface and a bridge share the same MAC address, which is often the case, then using that MAC address results in an error due to ambiguity, and you must resort to a name instead).
1 5 .1 5 .1 .1 . De fining and st art ing a ho st physical m achine int e rface via an XML file
The vi rsh i face-d efi ne file command define a host interface from an XML file. This command will only define the interface and will not start it.
vi rsh i face-d efi ne iface.xml
To start an interface which has already been defined, run i face-start interface, where interface is the interface name.
1 5 .1 5 .1 .2 . Edit ing t he XML co nfigurat io n file fo r t he ho st int e rface
The command i face-ed i t interface edits the XML configuration file for a host interface. This is
for more information about these files.)
1 5 .1 5 .1 .3. List ing act ive ho st int e rface s
The i face-l i st --i nacti ve --al l displays a list of active host interfaces. If --al l is specified, this list will also include interfaces that are defined but are inactive. If --i nacti ve is specified only the inactive interfaces will be listed.
1 5 .1 5 .1 .4 . Co nve rt ing a MAC addre ss int o an int e rface nam e
The i face-name interface command converts a host interface MAC to an interface name, provided the MAC address is unique among the host’s interfaces. This command requires interface which is the interface's MAC address.
The i face-mac interface command will convert a host's interface name to MAC address where in this case interface, is the interface name.
1 5 .1 5 .1 .5 . St o pping a spe cific ho st physical m achine int e rface
The vi rsh i face-d estro y interface command destroys (stops) a given host interface, which is the same as running i f-d o wn on the host. This command will disable that interface from active use and takes effect immediately.
To undefine the interface, use the i face-und efi ne interface command along with the interface name.
1 5 .1 5 .1 .6 . Displaying t he ho st co nfigurat io n file
vi rsh i face-d umpxml interface --i nacti ve displays the host interface information as an
XML dump to stdout. If the --i nacti ve argument is specified, then the output reflects the persistent state of the interface that will be used the next time it is started.
1 5 .1 5 .1 .7 . Cre at ing bridge de vice s
The i face-bri d g e creates a bridge device named bridge, and attaches the existing network device interface to the new bridge, which starts working immediately, with STP enabled and a delay of 0.
204
Chapt er 1 5. Managing guest virt ual machines wit h virsh
# vi rsh i face-bri d g e interface bridge --no -stp delay --no -start
Note that these settings can be altered with --no-stp, --no-start, and an integer number of seconds for delay. All IP address configuration of interface will be moved to the new bridge device. Refer to
Section 15.15.1.8, “ Tearing down a bridge device” for information on tearing down the bridge.
1 5 .1 5 .1 .8 . T e aring do wn a bridge de vice
The i face-unbri d g e bridge --no -start command tears down a specified bridge device named bridge, releases its underlying interface back to normal usage, and moves all IP address configuration from the bridge device to the underlying device. The underlying interface is restarted unless --no -start argument is used, but keep in mind not restarting is generally not recommended.
Refer to
Section 15.15.1.7, “ Creating bridge devices”
for the command to use to create a bridge.
1 5 .1 5 .1 .9 . Manipulat ing int e rface snapsho t s
The i face-beg i n command creates a snapshot of current host interface settings, which can later be committed (with i face-co mmi t) or restored (i face-ro l l back). If a snapshot already exists, then this command will fail until the previous snapshot has been committed or restored. Undefined behavior will result if any external changes are made to host interfaces outside of the libvirt API between the time of the creation of a snapshot and its eventual commit or rollback.
Use the i face-co mmi t command to declare all changes made since the last i face-beg i n as working, and then delete the rollback point. If no interface snapshot has already been started via
i face-beg i n, then this command will fail.
Use the i face-ro l l back to revert all host interface settings back to the state that recorded the last time the i face-beg i n command was executed. If i face-beg i n command had not been previously executed, then i face-ro l l back will fail. Note that rebooting the host physical machine also serves as an implicit rollback point.
15.15.2. Managing snapshot s
The sections that follow describe actions that can be done in order to manipulate domain snapshots.
Snapshots take the disk, memory, and device state of a domain at a specified point-in-time, and save it for future use. Snapshots have many uses, from saving a "clean" copy of an OS image to saving a domain’s state before what may be a potentially destructive operation. Snapshots are identified with a unique name. See the libvirt website for documentation of the XML format used to represent properties of snapshots.
1 5 .1 5 .2 .1 . Cre at ing Snapsho t s
The vi rsh snapsho t create command creates a snapshot for domain with the properties specified in the domain XML file (such as <name> and <description> elements, as well as <disks>).
To create a snapshot, run:
# snapsho t-create <d o mai n> <xml fi l e> [--red efi ne] [--current] [--no -
metad ata] [--reuse-external ]
The domain name, ID, or UID may be used as the domain requirement. The XML requirement is a string that must in the very least contain the <name>, <description> and <disks> elements.
205
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
Live snapshots are not supported in Red Hat Enterprise Linux. There are additional arguments available with the vi rsh snapsho t create command for use with live snapshots which are visible in libvirt, but not supported in Red Hat Enterprise Linux 6.
The arguments available in Red Hat Enterprise Linux include:
--red efi ne specifies that if all XML elements produced by snapsho t-d umpxml are valid; it can be used to migrate snapshot hierarchy from one machine to another, to recreate hierarchy for the case of a transient domain that goes away and is later recreated with the same name and UUID, or to make slight alterations in the snapshot metadata (such as host-specific aspects of the domain XML embedded in the snapshot). When this flag is supplied, the xml fi l e argument is mandatory, and the domain’s current snapshot will not be altered unless the --current flag is also given.
--no -metad ata creates the snapshot, but any metadata is immediately discarded (that is, libvirt does not treat the snapshot as current, and cannot revert to the snapshot unless --red efi ne is later used to teach libvirt about the metadata again).
--reuse-external , if used and snapshot XML requests an external snapshot with a destination of an existing file, then the destination must exist, and is reused; otherwise, a snapshot is refused to avoid losing contents of the existing files.
1 5 .1 5 .2 .2 . Cre at ing a snapsho t fo r t he curre nt do m ain
The vi rsh snapsho t-create-as d o mai n command creates a snapshot for the domain with the properties specified in the domain XML file (such as <name> and <description> elements). If these values are not included in the XML string, libvirt will choose a value. To create a snapshot run:
#snapsho t-create-as d o mai n {[--pri nt-xml ] | [--no -metad ata] [--reuse-
external ]} [name] [d escri pti o n] [--d i skspec] d i skspec]
The remaining optional arguments are as follows:
--pri nt-xml creates appropriate XML for snapsho t-create as output, rather than actually creating a snapshot.
--d i skspec option can be used to control how --d i sk-o nl y and external checkpoints create external files. This option can occur multiple times, according to the number of <disk> elements in the domain XML. Each <diskspec> is in the form disk[,snapsho t= type][,d ri ver= type]
[,fi l e= name]. To include a literal comma in disk or in fi l e= name, escape it with a second comma. A literal --d i skspec must precede each diskspec unless all three of <domain>,
<name>, and <description> are also present. For example, a diskspec of
vd a,snapsho t= external ,fi l e= /path/to ,,new results in the following XML:
<disk name=’vda’ snapshot=’external’>
<source file=’/path/to,new’/>
</disk>
--reuse-external is specified, and the domain XML or diskspec option requests an external snapshot with a destination of an existing file, then the destination must exist, and is reused; otherwise, a snapshot is refused to avoid losing contents of the existing files.
206
Chapt er 1 5. Managing guest virt ual machines wit h virsh
--no -metad ata creates snapshot data but any metadata is immediately discarded (that is, libvirt does not treat the snapshot as current, and cannot revert to the snapshot unless snapshot-create is later used to teach libvirt about the metadata again). This flag is incompatible with --pri nt-
xml .
1 5 .1 5 .2 .3. T aking a snapsho t o f t he curre nt do m ain
This command is used to query which snapshot is currently in use. To use, run:
# vi rsh snapsho t-current d o mai n {[--name] | [--securi ty-i nfo ] |
[snapsho tname]}
If snapsho tname is not used, snapshot XML for the domain’s current snapshot (if there is one) will be displayed as output. If --name is specified, just the current snapshot name instead of the full XML will be sent as output. If --securi ty-i nfo is supplied, security sensitive information will be included in the XML. Using snapsho tname, libvirt generates a request to make the existing named snapshot become the current snapshot, without reverting it to the domain.
1 5 .1 5 .2 .4 . snapsho t -e dit -do m ain
This command is used to edit the snapshot that is currently in use. To use, run:
#vi rsh snapsho t-ed i t d o mai n [snapsho tname] [--current] {[--rename] [--
cl o ne]}
If both snapsho tname and --current are specified, it forces the edited snapshot to become the current snapshot. If snapsho tname is omitted, then --current must be supplied, in order to edit the current snapshot.
This is equivalent to the following command sequence below, but it also includes some error checking:
# virsh snapshot-dumpxml dom name > snapshot.xml
# vi snapshot.xml [note - this can be any editor]
# virsh snapshot-create dom snapshot.xml --redefine [--current]
If --rename is specified, then the resulting edited file gets saved in a different file name. If --cl o ne is specified, then changing the snapshot name will create a clone of the snapshot metadata. If neither is specified, then the edits will not change the snapshot name. Note that changing a snapshot name must be done with care, since the contents of some snapshots, such as internal snapshots within a single qcow2 file, are accessible only from the original snapshot filename.
1 5 .1 5 .2 .5 . snapsho t -info -do m ain
snapsho t-i nfo -d o mai n displays information about the snapshots. To use, run:
# snapsho t-i nfo d o mai n {snapsho t | --current}
Outputs basic information about a specified snapsho t , or the current snapshot with --current.
1 5 .1 5 .2 .6 . snapsho t -list -do m ain
List all of the available snapshots for the given domain, defaulting to show columns for the snapshot name, creation time, and domain state. To use, run:
207
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
#vi rsh snapsho t-l i st d o mai n [{--parent | --ro o ts | --tree}] [{[--fro m]
snapsho t | --current} [--d escend ants]] [--metad ata] [--no -metad ata] [-l eaves] [--no -l eaves] [--i nacti ve] [--acti ve] [--i nternal ] [--external ]
The remaining optional arguments are as follows:
--parent adds a column to the output table giving the name of the parent of each snapshot. This option may not be used with --ro o ts or --tree.
--ro o ts filters the list to show only the snapshots that have no parents. This option may not be used with --parent or --tree.
--tree displays output in a tree format, listing just snapshot names. These three options are mutually exclusive. This option may not be used with --ro o ts or --parent.
--fro m filters the list to snapshots which are children of the given snapshot; or if --current is provided, will cause the list to start at the current snapshot. When used in isolation or with --
parent, the list is limited to direct children unless --d escend ants is also present. When used with --tree, the use of --d escend ants is implied. This option is not compatible with --ro o ts.
Note that the starting point of --fro m or --current is not included in the list unless the --tree option is also present.
--l eaves is specified, the list will be filtered to just snapshots that have no children. Likewise, if -
-no -l eaves is specified, the list will be filtered to just snapshots with children. (Note that omitting both options does no filtering, while providing both options will either produce the same list or error out depending on whether the server recognizes the flags) Filtering options are not compatible with --tree..
--metad ata is specified, the list will be filtered to just snapshots that involve libvirt metadata, and thus would prevent the undefining of a persistent domain, or be lost on destroy of a transient domain. Likewise, if --no -metad ata is specified, the list will be filtered to just snapshots that exist without the need for libvirt metadata.
--i nacti ve is specified, the list will be filtered to snapshots that were taken when the domain was shut off. If --acti ve is specified, the list will be filtered to snapshots that were taken when the domain was running, and where the snapshot includes the memory state to revert to that running state. If --d i sk-o nl y is specified, the list will be filtered to snapshots that were taken when the domain was running, but where the snapshot includes only disk state.
--i nternal is specified, the list will be filtered to snapshots that use internal storage of existing disk images. If --external is specified, the list will be filtered to snapshots that use external files for disk images or memory state.
1 5 .1 5 .2 .7 . snapsho t -dum pxm l do m ain snapsho t
vi rsh snapsho t-d umpxml d o mai n snapsho t outputs the snapshot XML for the domain’s snapshot named snapshot. To use, run:
# vi rsh snapsho t-d umpxml d o mai n snapsho t [--securi ty-i nfo ]
The --securi ty-i nfo option will also include security sensitive information. Use snapsho t-
current to easily access the XML of the current snapshot.
1 5 .1 5 .2 .8 . snapsho t -pare nt do m ain
208
Chapt er 1 5. Managing guest virt ual machines wit h virsh
Outputs the name of the parent snapshot, if any, for the given snapshot, or for the current snapshot with --current. To use, run:
#vi rsh snapsho t-parent d o mai n {snapsho t | --current}
1 5 .1 5 .2 .9 . snapsho t -re ve rt do m ain
Reverts the given domain to the snapshot specified by snapsho t, or to the current snapshot with --
current.
Warning
Be aware that this is a destructive action; any changes in the domain since the last snapshot was taken will be lost. Also note that the state of the domain after snapsho t-revert is complete will be the state of the domain at the time the original snapshot was taken.
To revert the snapshot, run
# snapsho t-revert d o mai n {snapsho t | --current} [{--runni ng | --paused }]
[--fo rce]
Normally, reverting to a snapshot leaves the domain in the state it was at the time the snapshot was created, except that a disk snapshot with no guest virtual machine state leaves the domain in an inactive state. Passing either the --runni ng or --paused flag will perform additional state changes
(such as booting an inactive domain, or pausing a running domain). Since transient domains cannot be inactive, it is required to use one of these flags when reverting to a disk snapshot of a transient domain.
There are two cases where a snapsho t revert involves extra risk, which requires the use of --
fo rce to proceed. One is the case of a snapshot that lacks full domain information for reverting configuration; since libvirt cannot prove that the current configuration matches what was in use at the time of the snapshot, supplying --fo rce assures libvirt that the snapshot is compatible with the current configuration (and if it is not, the domain will likely fail to run). The other is the case of reverting from a running domain to an active state where a new hypervisor has to be created rather than reusing the existing hypervisor, because it implies drawbacks such as breaking any existing
VNC or Spice connections; this condition happens with an active snapshot that uses a provably incompatible configuration, as well as with an inactive snapshot that is combined with the --start or --pause flag.
1 5 .1 5 .2 .1 0 . snapsho t -de le t e do m ain
snapsho t-d el ete d o mai n deletes the snapshot for the specified domain. To do this, run:
# vi rsh snapsho t-d el ete d o mai n {snapsho t | --current} [--metad ata] [{--
chi l d ren | --chi l d ren-o nl y}]
This command Deletes the snapshot for the domain named snapsho t, or the current snapshot with
--current. If this snapshot has child snapshots, changes from this snapshot will be merged into the children. If the option --chi l d ren is used, then it will delete this snapshot and any children of this snapshot. If --chi l d ren-o nl y is used, then it will delete any children of this snapshot, but leave this snapshot intact. These two flags are mutually exclusive.
209
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The --metad ata is used it will delete the snapshot's metadata maintained by libvirt, while leaving the snapshot contents intact for access by external tools; otherwise deleting a snapshot also removes its data contents from that point in time.
15.16. Guest virt ual machine CPU model configurat ion
15.16.1. Int roduct ion
Every hypervisor has its own policy for what a guest virtual machine will see for its CPUs by default.
Whereas some hypervisors decide which CPU host physical machine features will be available for the guest virtual machine, QEMU/KVM presents the guest virtual machine with a generic model named
q emu32 or q emu6 4 . These hypervisors perform more advanced filtering, classifying all physical
CPUs into a handful of groups and have one baseline CPU model for each group that is presented to the guest virtual machine. Such behavior enables the safe migration of guest virtual machines between host physical machines, provided they all have physical CPUs that classify into the same group. libvirt does not typically enforce policy itself, rather it provides the mechanism on which the higher layers define their own desired policy. Understanding how to obtain CPU model information and define a suitable guest virtual machine CPU model is critical to ensure guest virtual machine migration is successful between host physical machines. Note that a hypervisor can only emulate features that it is aware of and features that were created after the hypervisor was released may not be emulated.
15.16.2. Learning about t he host physical machine CPU model
The vi rsh capabi l i ti es command displays an XML document describing the capabilities of the hypervisor connection and host physical machine. The XML schema displayed has been extended to provide information about the host physical machine CPU model. One of the big challenges in describing a CPU model is that every architecture has a different approach to exposing their capabilities. On x86, the capabilities of a modern CPU are exposed via the CPUID instruction.
Essentially this comes down to a set of 32-bit integers with each bit given a specific meaning.
Fortunately AMD and Intel agree on common semantics for these bits. Other hypervisors expose the notion of CPUID masks directly in their guest virtual machine configuration format. However,
QEMU/KVM supports far more than just the x86 architecture, so CPUID is clearly not suitable as the canonical configuration format. QEMU ended up using a scheme which combines a CPU model name string, with a set of named flags. On x86, the CPU model maps to a baseline CPUID mask, and the flags can be used to then toggle bits in the mask on or off. libvirt decided to follow this lead and uses a combination of a model name and flags.
It is not practical to have a database listing all known CPU models, so libvirt has a small list of baseline CPU model names. It chooses the one that shares the greatest number of CPUID bits with the actual host physical machine CPU and then lists the remaining bits as named features. Notice that libvirt does not display which features the baseline CPU contains. This might seem like a flaw at first, but as will be explained in this section, it is not actually necessary to know this information.
15.16.3. Det ermining a compat ible CPU model t o suit a pool of host physical machines
Now that it is possible to find out what CPU capabilities a single host physical machine has, the next step is to determine what CPU capabilities are best to expose to the guest virtual machine. If it is known that the guest virtual machine will never need to be migrated to another host physical machine, the host physical machine CPU model can be passed straight through unmodified. A virtualized data center may have a set of configurations that can guarantee all servers will have
100% identical CPUs. Again the host physical machine CPU model can be passed straight through unmodified. The more common case, though, is where there is variation in CPUs between host
210
Chapt er 1 5. Managing guest virt ual machines wit h virsh
physical machines. In this mixed CPU environment, the lowest common denominator CPU must be determined. This is not entirely straightforward, so libvirt provides an API for exactly this task. If libvirt is provided a list of XML documents, each describing a CPU model for a host physical machine, libvirt will internally convert these to CPUID masks, calculate their intersection, and convert the
CPUID mask result back into an XML CPU description.
Here is an example of what libvirt reports as the capabilities on a basic workstation, when the vi rsh
capabi l i ti esis executed:
<capabilities>
<host>
<cpu>
<arch> i686 </arch>
<model> pentium3 </model>
<topology sockets= '1' cores= '2' threads= '1' />
<feature name= 'lahf_lm' />
<feature name= 'lm' />
<feature name= 'xtpr' />
<feature name= 'cx16' />
<feature name= 'ssse3' />
<feature name= 'tm2' />
<feature name= 'est' />
<feature name= 'vmx' />
<feature name= 'ds_cpl' />
<feature name= 'monitor' />
<feature name= 'pni' />
<feature name= 'pbe' />
<feature name= 'tm' />
<feature name= 'ht' />
<feature name= 'ss' />
<feature name= 'sse2' />
<feature name= 'acpi' />
<feature name= 'ds' />
<feature name= 'clflush' />
<feature name= 'apic' />
</cpu>
</host>
</capabilities>
Fig u re 15.3. Pu llin g h o st p h ysical mach in e' s CPU mo d el in f o rmat io n
Now compare that to any random server, with the same vi rsh capabi l i ti es command:
<capabilities>
<host>
<cpu>
<arch> x86_64 </arch>
211
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<model> phenom </model>
<topology sockets= '2' cores= '4' threads= '1' />
<feature name= 'osvw' />
<feature name= '3dnowprefetch' />
<feature name= 'misalignsse' />
<feature name= 'sse4a' />
<feature name= 'abm' />
<feature name= 'cr8legacy' />
<feature name= 'extapic' />
<feature name= 'cmp_legacy' />
<feature name= 'lahf_lm' />
<feature name= 'rdtscp' />
<feature name= 'pdpe1gb' />
<feature name= 'popcnt' />
<feature name= 'cx16' />
<feature name= 'ht' />
<feature name= 'vme' />
</cpu>
...snip...
Fig u re 15.4 . G en erat e CPU d escrip t io n f ro m a ran d o m server
To see if this CPU description is compatible with the previous workstation CPU description, use the
vi rsh cpu-co mpare command.
The reduced content was stored in a file named vi rsh-caps-wo rkstati o n-cpu-o nl y. xml and the vi rsh cpu-co mpare command can be executed on this file:
# virsh cpu-compare virsh-caps-workstation-cpu-only.xml
Host physical machine CPU is a superset of CPU described in virsh-capsworkstation-cpu-only.xml
As seen in this output, libvirt is correctly reporting that the CPUs are not strictly compatible. This is because there are several features in the server CPU that are missing in the client CPU. To be able to migrate between the client and the server, it will be necessary to open the XML file and comment out some features. To determine which features need to be removed, run the vi rsh cpu-basel i ne command, on the bo th-cpus. xml which contains the CPU information for both machines. Running
# vi rsh cpu-basel i ne bo th-cpus. xml , results in:
<cpu match= 'exact' >
<model> pentium3 </model>
<feature policy= 'require' name= 'lahf_lm' />
<feature policy= 'require' name= 'lm' />
<feature policy= 'require' name= 'cx16' />
<feature policy= 'require' name= 'monitor' />
<feature policy= 'require' name= 'pni' />
<feature policy= 'require' name= 'ht' />
<feature policy= 'require' name= 'sse2' />
<feature policy= 'require' name= 'clflush' />
<feature policy= 'require' name= 'apic' />
</cpu>
212
Chapt er 1 5. Managing guest virt ual machines wit h virsh
Fig u re 15.5. Co mp o sit e CPU b aselin e
This composite file shows which elements are in common. Everything that is not in common should be commented out.
15.17. Configuring t he guest virt ual machine CPU model
For simple defaults, the guest virtual machine CPU configuration accepts the same basic XML representation as the host physical machine capabilities XML exposes. In other words, the XML from the cpu-basel i ne virsh command can now be copied directly into the guest virtual machine XML at the top level under the <domain> element. In the previous XML snippet, there are a few extra attributes available when describing a CPU in the guest virtual machine XML. These can mostly be ignored, but for the curious here is a quick description of what they do. The top level <cpu> element has an attribute called match with possible values of: match='minimum' - the host physical machine CPU must have at least the CPU features described in the guest virtual machine XML. If the host physical machine has additional features beyond the guest virtual machine configuration, these will also be exposed to the guest virtual machine.
match='exact' - the host physical machine CPU must have at least the CPU features described in the guest virtual machine XML. If the host physical machine has additional features beyond the guest virtual machine configuration, these will be masked out from the guest virtual machine.
match='strict' - the host physical machine CPU must have exactly the same CPU features described in the guest virtual machine XML.
The next enhancement is that the <feature> elements can each have an extra 'policy' attribute with possible values of: policy='force' - expose the feature to the guest virtual machine even if the host physical machine does not have it. This is usually only useful in the case of software emulation.
policy='require' - expose the feature to the guest virtual machine and fail if the host physical machine does not have it. This is the sensible default.
policy='optional' - expose the feature to the guest virtual machine if it happens to support it.
policy='disable' - if the host physical machine has this feature, then hide it from the guest virtual machine.
policy='forbid' - if the host physical machine has this feature, then fail and refuse to start the guest virtual machine.
The 'forbid' policy is for a niche scenario where an incorrectly functioning application will try to use a feature even if it is not in the CPUID mask, and you wish to prevent accidentally running the guest virtual machine on a host physical machine with that feature. The 'optional' policy has special behavior with respect to migration. When the guest virtual machine is initially started the flag is optional, but when the guest virtual machine is live migrated, this policy turns into 'require', since you cannot have features disappearing across migration.
15.18. Managing resources for guest virt ual machines
213
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
virsh allows the grouping and allocation of resources on a per guest virtual machine basis. This is managed by the libvirt daemon which creates cgroups and manages them on behalf of the guest virtual machine. The only thing that is left for the system administrator to do is to either query or set tunables against specified guest virtual machines. The following tunables may used:
memo ry - The memory controller allows for setting limits on RAM and swap usage and querying cumulative usage of all processes in the group
cpuset - The CPU set controller binds processes within a group to a set of CPUs and controls migration between CPUs.
cpuacct - The CPU accounting controller provides information about CPU usage for a group of processes.
cpu -The CPU scheduler controller controls the prioritization of processes in the group. This is similar to granting ni ce level privileges.
d evi ces - The devices controller grants access control lists on character and block devices.
freezer - The freezer controller pauses and resumes execution of processes in the group. This is similar to SIG ST O P for the whole group.
net_cl s - The network class controller manages network utilization by associating processes with a tc network class.
In creating a group hierarchy cgroup will leave mount point and directory setup entirely to the administrators’ discretion and is more complex than just adding some mount points to /etc/fstab.
It is necessary to setup the directory hierarchy and decide how processes get placed within it. This can be done with the following virsh commands:
sched i nfo - described in
Section 15.19, “ Setting schedule parameters”
bl kd evi o tune - described in
Section 15.20, “ Disk I/O throttling”
bl ki o tune- described in
Section 15.21, “ Display or set block I/O parameters”
d o mi ftune- described in
Section 15.5.9, “ Setting network interface bandwidth parameters”
memtune - described in
Section 15.22, “ Configuring memory Tuning”
15.19. Set t ing schedule paramet ers
sched i nfo allows scheduler parameters to be passed to guest virtual machines. The following command format should be used:
#vi rsh sched i nfo domain --set --wei g ht --cap --current --co nfi g --l i ve
Each parameter is explained below:
d o mai n - this is the guest virtual machine domain
--set - the string placed here is the controller or action that is to be called. Additional parameters or values if required should be added as well.
--current - when used with --set, will use the specified set string as the current scheduler information. When used without will display the current scheduler information.
--co nfi g - - when used with --set, will use the specified set string on the next reboot. When used without will display the scheduler information that is saved in the configuration file.
214
Chapt er 1 5. Managing guest virt ual machines wit h virsh
--l i ve - when used with --set, will use the specified set string on a guest virtual machine that is currently running. When used without will display the configuration setting currently used by the running virtual machine
The scheduler can be set with any of the following parameters: cpu_shares, vcpu_period and
vcpu_quota.
Examp le 15.5. sch ed in f o sh o w
This example shows the shell guest virtual machine's schedule information
# virsh schedinfo shell
Scheduler : posix cpu_shares : 1024 vcpu_period : 100000 vcpu_quota : -1
Examp le 15.6 . sch ed in f o set
In this example, the cpu_shares is changed to 2046. This effects the current state and not the configuration file.
# virsh schedinfo --set cpu_shares=2046 shell
Scheduler : posix cpu_shares : 2046 vcpu_period : 100000 vcpu_quota : -1
15.20. Disk I/O t hrot t ling
vi rsh bl kd evi o tune sets disk I/O throttling for a specified guest virtual machine. This can prevent a guest virtual machine from over utilizing shared resources and thus impacting the performance of other guest virtual machines. The following format should be used:
# vi rsh bl kd evi o tune <d o mai n> <d evi ce> [[--co nfi g ] [--l i ve] | [--
current]] [[to tal -bytes-sec] | [read -bytes-sec] [wri te-bytes-sec]]
[[to tal -i o ps-sec] [read -i o ps-sec] [wri te-i o ps-sec]]
The only required parameter is the domain name of the guest virtual machine. To list the domain name, run the d o mbl kl i st command. The --co nfi g , --l i ve, and --current arguments function the same as in
Section 15.19, “ Setting schedule parameters” . If no limit is specified, it will
query current I/O limits setting. Otherwise, alter the limits with the following flags:
--to tal -bytes-sec - specifies total throughput limit in bytes per second.
--read -bytes-sec - specifies read throughput limit in bytes per second.
--wri te-bytes-sec - specifies write throughput limit in bytes per second.
--to tal -i o ps-sec - specifies total I/O operations limit per second.
215
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
--read -i o ps-sec - specifies read I/O operations limit per second.
--wri te-i o ps-sec - specifies write I/O operations limit per second.
For more information refer to the blkdeviotune section of the virsh MAN page. For an example domain
XML refer to Figure 21.24, “ Devices - Hard drives, floppy disks, CDROMs” .
15.21. Display or set block I/O paramet ers
bl ki o tune sets and or displays the I/O parameters for a specified guest virtual machine. The following format should be used:
# vi rsh bl ki o tune d o mai n [--wei g ht wei g ht] [--d evi ce-wei g hts d evi ce-
wei g hts] [[--co nfi g ] [--l i ve] | [--current]]
More information on this command can be found in the Virtualization Tuning and Optimization Guide
15.22. Configuring memory T uning
The vi rsh memtune vi rtual _machi ne --parameter si ze is covered in the Virtualization
Tuning and Opitimization Guide.
15.23. Virt ual Net working Commands
The following commands manipulate virtual networks. libvirt has the capability to define virtual networks which can then be used by domains and linked to actual network devices. For more detailed information about this feature see the documentation at libvirt's website . Many of the commands for virtual networks are similar to the ones used for domains, but the way to name a virtual network is either by its name or UUID.
15.23.1. Aut ost art ing a virt ual net work
This command will configure a virtual network to be started automatically when the guest virtual machine boots. To run this command:
# vi rsh net-auto start network [--d i sabl e]
This command accepts the --d i sabl e option which disables the autostart command.
15.23.2. Creat ing a virt ual net work from an XML file
This command creates a virtual network from an XML file. Refer to libvirt's website to get a description of the XML network format used by libvirt. In this command file is the path to the XML file. To create the virtual network from an XML file, run:
# vi rsh net-create file
15.23.3. Defining a virt ual net work from an XML file
This command defines a virtual network from an XML file, the network is just defined but not instantiated. To define the virtual network, run:
216
Chapt er 1 5. Managing guest virt ual machines wit h virsh
# net-d efi ne file
15.23.4 . St opping a virt ual net work
This command destroys (stops) a given virtual network specified by its name or UUID. This takes effect immediately. To stop the specified network network is required.
# net-d estro y network
15.23.5. Creat ing a dump file
This command outputs the virtual network information as an XML dump to stdout for the specified virtual network. If --i nacti ve is specified, then physical functions are not expanded into their associated virtual functions. To create the dump file, run:
# vi rsh net-d umpxml network [--i nacti ve]
15.23.6. Eding a virt ual net work's XML configurat ion file
This command edits the XML configuration file for a network. This is equivalent to:
#vi rsh net-d umpxml --i nacti ve network > network.xml vi network.xml (or make changes with your other text editor) virsh net-define network.xml
except that it does some error checking. The editor used can be supplied by the $VISUAL or
$EDITOR environment variables, and defaults to "vi". To edit the network, run:
#vi rsh net-ed i t network
15.23.7. Get t ing informat ion about a virt ual net work
This command returns basic information about the network object. To get the network information, run:
# vi rsh net-i nfo network
15.23.8. List ing informat ion about a virt ual net work
Returns the list of active networks, if --al l is specified this will also include defined but inactive networks, if --i nacti ve is specified only the inactive ones will be listed. You may also want to filter the returned networks by --persi stent to list the persitent ones, --transi ent to list the transient ones, --auto start to list the ones with autostart enabled, and --no -auto start to list the ones with autostart disabled.
Note: When talking to older servers, this command is forced to use a series of API calls with an inherent race, where a pool might not be listed or might appear more than once if it changed state between calls while the list was being collected. Newer servers do not have this problem.
To list the virtual networks, run:
217
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# net-l i st [--i nacti ve | --al l ] [--persi stent] [<--transi ent>] [--
auto start] [<--no -auto start>]
15.23.9. Convert ing a net work UUID t o net work name
This command converts a network UUID to network name. To do this run:
# vi rsh net-name network-UUID
15.23.10. St art ing a (previously defined) inact ive net work
This command starts a (previously defined) inactive network. To do this, run:
# vi rsh net-start network
15.23.11. Undefining t he configurat ion for an inact ive net work
This command undefines the configuration for an inactive network. To do this, run:
# net-und efi ne network
15.23.12. Convert ing a net work name t o net work UUID
This command converts a network name to network UUID. To do this, run:
# vi rsh net-uui d network-name
15.23.13. Updat ing an exist ing net work definit ion file
This command updates the given section of an existing network definition, taking effect immediately, without needing to destroy and re-start the network. This command is one of "add-first", "add-last",
"add" (a synonym for add-last), "delete", or "modify". section is one of ""bridge", "domain", "ip", "ipdhcp-host", "ip-dhcp-range", "forward", "forward-interface", "forward-pf", "portgroup", "dns-host",
"dns-txt", or "dns-srv", each section being named by a concatenation of the xml element hierarchy leading to the element being changed. For example, "ip-dhcp-host" will change a <ho st> element that is contained inside a <d hcp> element inside an <i p> element of the network. xml is either the text of a complete xml element of the type being changed (e.g. "<host mac="00:11:22:33:44:55’ ip=’192.0.2.1’/>", or the name of a file that contains a complete xml element. Disambiguation is done by looking at the first character of the provided text - if the first character is "<", it is xml text, if the first character is not ">", it is the name of a file that contains the xml text to be used. The --parent-index option is used to specify which of several parent elements the requested element is in (0-based). For example, a dhcp <ho st> element could be in any one of multiple <i p> elements in the network; if a parent-index isn’t provided, the "most appropriate" <i p> element will be selected (usually the only one that already has a <d hcp> element), but if --parent-index is given, that particular instance of
<i p> will get the modification. If --live is specified, affect a running network. If --config is specified, affect the next startup of a persistent network. If -- current is specified, affect the current network state. Both --live and --config flags may be given, but --current is exclusive. Not specifying any flag is the same as specifying --current.
To update the configuration file, run:
218
Chapt er 1 5. Managing guest virt ual machines wit h virsh
# vi rsh net-upd ate network command section xml [--parent-index index]
[[--live] [--config] | [--current]]
219
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 16. Managing guests with the Virtual Machine Manager
(virt-manager)
This section describes the Virtual Machine Manager (vi rt-manag er) windows, dialog boxes, and various GUI controls.
vi rt-manag er provides a graphical view of hypervisors and guests on your host system and on remote host systems. vi rt-manag er can perform virtualization management tasks, including: defining and creating guests, assigning memory, assigning virtual CPUs, monitoring operational performance, saving and restoring, pausing and resuming, and shutting down and starting guests, links to the textual and graphical consoles, and live and offline migrations.
16.1. St art ing virt -manager
To start vi rt-manag er session open the Ap p licat io n s menu, then the Syst em T o o ls menu and select Virt u al Mach in e Man ag er (vi rt-manag er).
The vi rt-manag er main window appears.
Fig u re 16 .1. St art in g vi rt-manag er
Alternatively, vi rt-manag er can be started remotely using ssh as demonstrated in the following command: ssh -X host's address
[remotehost]# virt-manager
220
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
16.2. T he Virt ual Machine Manager main window
This main window displays all the running guests and resources used by guests. Select a guest by double clicking the guest's name.
Fig u re 16 .2. Virt u al Mach in e Man ag er main win d o w
16.3. T he virt ual hardware det ails window
The virtual hardware details window displays information about the virtual hardware configured for the guest. Virtual hardware resources can be added, removed and modified in this window. To access the virtual hardware details window, click on the icon in the toolbar.
221
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .3. T h e virt u al h ard ware d et ails ico n
Clicking the icon displays the virtual hardware details window.
222
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .4 . T h e virt u al h ard ware d et ails win d o w
16.3.1. At t aching USB devices t o a guest virt ual machine
Note
In order to attach the USB device to the guest virtual machine, you first must attach it to the host physical machine and confirm that the device is working. If the guest is running, you need to shut it down before proceeding.
Pro ced u re 16 .1. At t ach in g USB d evices u sin g Virt - Man ag er
1. Open the guest virtual machine's Virtual Machine Details screen.
2. Click Ad d Hard ware
223
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
224
Fig u re 16 .5. Ad d Hard ware Bu t t o n
3. In the Ad d New Virt u al Hard ware popup, select USB Ho st Device, select the device you want to attach from the list and Click Fi ni sh.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .6 . Ad d USB Device
4. To use the USB device in the guest virtual machine, start the guest virtual machine.
16.4 . Virt ual Machine graphical console
This window displays a guest's graphical console. Guests can use several different protocols to export their graphical framebuffers: vi rt-manag er supports VNC and SPICE. If your virtual machine is set to require authentication, the Virtual Machine graphical console prompts you for a password before the display appears.
225
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .7. G rap h ical co n so le win d o w
Note
VNC is considered insecure by many security experts, however, several changes have been made to enable the secure usage of VNC for virtualization on Red Hat enterprise Linux. The guest machines only listen to the local host's loopback address (127. 0 . 0 . 1). This ensures only those with shell privileges on the host can access virt-manager and the virtual machine through VNC. Although virt-manager is configured to listen to other public network interfaces and alternative methods can be configured, it is not recommended.
Remote administration can be performed by tunneling over SSH which encrypts the traffic.
Although VNC can be configured to access remotely without tunneling over SSH, for security reasons, it is not recommended. To remotely administer the guest follow the instructions in:
Chapter 6, Remote management of guests
. TLS can provide enterprise level security for managing guest and host systems.
226
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Your local desktop can intercept key combinations (for example, Ctrl+Alt+F1) to prevent them from being sent to the guest machine. You can use the Sen d key menu option to send these sequences.
From the guest machine window, click the Sen d key menu and select the key sequence to send. In addition, from this menu you can also capture the screen output.
SPICE is an alternative to VNC available for Red Hat Enterprise Linux.
16.5. Adding a remot e connect ion
This procedure covers how to set up a connection to a remote system using vi rt-manag er.
1. To create a new connection open the Fi l e menu and select the Ad d C o nnecti o n. . . menu item.
2. The Ad d C o nnecti o n wizard appears. Select the hypervisor. For Red Hat Enterprise Linux
6 systems select Q EMU/KVM. Select Local for the local system or one of the remote connection options and click C o nnect. This example uses Remote tunnel over SSH which works on default installations. For more information on configuring remote connections refer to
Chapter 6, Remote management of guests
Fig u re 16 .8. Ad d Co n n ect io n
3. Enter the root password for the selected host when prompted.
A remote host is now connected and appears in the main vi rt-manag er window.
227
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .9 . Remo t e h o st in t h e main virt - man ag er win d o w
16.6. Displaying guest det ails
You can use the Virtual Machine Monitor to view activity information for any virtual machines on your system.
To view a virtual system's details:
1. In the Virtual Machine Manager main window, highlight the virtual machine that you want to view.
228
Fig u re 16 .10. Select in g a virt u al mach in e t o d isp lay
2. From the Virtual Machine Manager Ed i t menu, select Vi rtual Machi ne D etai l s.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .11. Disp layin g t h e virt u al mach in e d et ails
When the Virtual Machine details window opens, there may be a console displayed. Should this happen, click Vi ew and then select D etai l s. The Overview window opens first by default. To go back to this window, select O vervi ew from the navigation pane on the left hand side.
The O vervi ew view shows a summary of configuration details for the guest.
229
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
230
Fig u re 16 .12. Disp layin g g u est d et ails o verview
3. Select P erfo rmance from the navigation pane on the left hand side.
The P erfo rmance view shows a summary of guest performance, including CPU and Memory usage.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .13. Disp layin g g u est p erf o rman ce d et ails
4. Select P ro cesso r from the navigation pane on the left hand side. The P ro cesso r view allows you to view or change the current processor allocation.
231
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
232
Fig u re 16 .14 . Pro cesso r allo cat io n p an el
5. Select Memo ry from the navigation pane on the left hand side. The Memo ry view allows you to view or change the current memory allocation.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .15. Disp layin g memo ry allo cat io n
6. Each virtual disk attached to the virtual machine is displayed in the navigation pane. Click on a virtual disk to modify or remove it.
233
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
234
Fig u re 16 .16 . Disp layin g d isk co n f ig u rat io n
7. Each virtual network interface attached to the virtual machine is displayed in the navigation pane. Click on a virtual network interface to modify or remove it.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .17. Disp layin g n et wo rk co n f ig u rat io n
16.7. Performance monit oring
Performance monitoring preferences can be modified with vi rt-manag er's preferences window.
To configure performance monitoring:
1. From the Ed i t menu, select P references.
Fig u re 16 .18. Mo d if yin g g u est p ref eren ces
The P references window appears.
2. From the Stats tab specify the time in seconds or stats polling options.
235
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .19 . Co n f ig u rin g p erf o rman ce mo n it o rin g
16.8. Displaying CPU usage for guest s
To view the CPU usage for all guests on your system:
1. From the Vi ew menu, select G raph, then the G uest C P U Usag e check box.
236
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .20. En ab lin g g u est CPU u sag e st at ist ics g rap h in g
2. The Virtual Machine Manager shows a graph of CPU usage for all virtual machines on your system.
Fig u re 16 .21. G u est CPU u sag e g rap h
16.9. Displaying CPU usage for host s
To view the CPU usage for all hosts on your system:
1. From the Vi ew menu, select G raph, then the Ho st C P U Usag e check box.
Fig u re 16 .22. En ab lin g h o st CPU u sag e st at ist ics g rap h in g
2. The Virtual Machine Manager shows a graph of host CPU usage on your system.
237
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .23. Ho st CPU u sag e g rap h
16.10. Displaying Disk I/O
To view the disk I/O for all virtual machines on your system:
1. Make sure that the Disk I/O statistics collection is enabled. To do this, from the Ed i t menu, select P references and click the Statstab.
2. Select the D i sk I/O checkbox.
238
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .24 . En ab lin g Disk I/O
3. To enable the Disk I.O display, from the Vi ew menu, select G raph, then the D i sk I/O check box.
Fig u re 16 .25. Select in g Disk I/O
4. The Virtual Machine Manager shows a graph of Disk I/O for all virtual machines on your system.
Fig u re 16 .26 . Disp layin g Disk I/O
16.11. Displaying Net work I/O
To view the network I/O for all virtual machines on your system:
1. Make sure that the Network I/O statistics collection is enabled. To do this, from the Ed i t menu, select P references and click the Statstab.
2. Select the Netwo rk I/O checkbox.
239
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
24 0
Fig u re 16 .27. En ab lin g Net wo rk I/O
3. To display the Network I/O statistics, from the Vi ew menu, select G raph, then the Netwo rk
I/O check box.
Chapt er 1 6 . Managing guest s wit h t he Virt ual Machine Manager (virt - manager)
Fig u re 16 .28. Select in g Net wo rk I/O
4. The Virtual Machine Manager shows a graph of Network I/O for all virtual machines on your system.
24 1
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 16 .29 . Disp layin g Net wo rk I/O
24 2
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
Chapter 17. Guest virtual machine disk access with offline tools
17.1. Int roduct ion
Red Hat Enterprise Linux 6 comes with tools to access, edit and create host physical machine disks or other disk images. There are several uses for these tools, including:
Viewing or downloading files located on a host physical machine disk.
Editing or uploading files onto a host physical machine disk.
Reading or writing host physical machine configuration.
Reading or writing the Windows Registry in Windows host physical machines.
Preparing new disk images containing files, directories, file systems, partitions, logical volumes and other options.
Rescuing and repairing host physical machines that fail to boot or those that need boot configuration changes.
Monitoring disk usage of host physical machines.
Auditing compliance of host physical machines, for example to organizational security standards.
Deploying host physical machines by cloning and modifying templates.
Reading CD and DVD ISO and floppy disk images.
Warning
You must n ever use these tools to write to a host physical machine or disk image which is attached to a running virtual machine, not even to open such a disk image in write mode.
Doing so will result in disk corruption of the guest virtual machine. The tools try to prevent you from doing this, however do not catch all cases. If there is any suspicion that a guest virtual machine might be running, it is strongly recommended that the tools not be used, or at least
always use the tools in read-only mode.
17.2. T erminology
This section explains the terms used throughout this chapter.
lib g u est f s ( G UEST FileSyst em LIBrary) - the underlying C library that provides the basic functionality for opening disk images, reading and writing files and so on. You can write C programs directly to this API, but it is quite low level.
g u est f ish ( G UEST Filesyst em In t eract ive SHell) is an interactive shell that you can use from the command line or from shell scripts. It exposes all of the functionality of the libguestfs API.
Various virt tools are built on top of libguestfs, and these provide a way to perform specific single tasks from the command line. Tools include virt - d f , virt - rescu e, virt - resiz e and virt - ed it .
24 3
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
h ivex and Au g eas are libraries for editing the Windows Registry and Linux configuration files respectively. Although these are separate from libguestfs, much of the value of libguestfs comes from the combination of these tools.
g u est mo u n t is an interface between libguestfs and FUSE. It is primarily used to mount file systems from disk images on your host physical machine. This functionality is not necessary, but can be useful.
17.3. Inst allat ion
To install libguestfs, guestfish, the libguestfs tools, guestmount and support for Windows guest virtual machines, subscribe to the RHEL V2WIN channel, go to the Red Hat Website and run the following command:
# yum install libguestfs guestfish libguestfs-tools libguestfs-mount libguestfs-winsupport
To install every libguestfs-related package including the language bindings, run the following command:
# yum install '*guestf*'
17.4 . T he guest fish shell
g u est f ish is an interactive shell that you can use from the command line or from shell scripts to access guest virtual machine file systems. All of the functionality of the libguestfs API is available from the shell.
To begin viewing or editing a virtual machine disk image, run the following command, substituting the path to your desired disk image: guestfish --ro -a /path/to/disk/image
- - ro means that the disk image is opened read-only. This mode is always safe but does not allow write access. Only omit this option when you are cert ain that the guest virtual machine is not running, or the disk image is not attached to a live guest virtual machine. It is not possible to use
l i bg uest vi rtual machi nefs to edit a live guest virtual machine, and attempting to will result in irreversible disk corruption.
/p at h /t o /d isk/imag e is the path to the disk. This can be a file, a host physical machine logical volume (such as /dev/VG/LV), a host physical machine device (/dev/cdrom) or a SAN LUN (/dev/sdf3).
Note
libguestfs and guestfish do not require root privileges. You only need to run them as root if the disk image being accessed needs root to read and/or write.
When you start guestfish interactively, it will display this prompt:
guestfish --ro -a /path/to/disk/image
24 4
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
Welcome to guestfish, the libguestfs filesystem interactive shell for editing virtual machine filesystems.
Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell
><fs>
At the prompt, type ru n to initiate the library and attach the disk image. This can take up to 30 seconds the first time it is done. Subsequent starts will complete much faster.
Note
libguestfs will use hardware virtualization acceleration such as KVM (if available) to speed up this process.
Once the ru n command has been entered, other commands can be used, as the following section demonstrates.
17.4 .1. Viewing file syst ems wit h guest fish
1 7 .4 .1 .1 . Manual list ing and vie wing
The l i st-fi l esystems command will list file systems found by libguestfs. This output shows a
Red Hat Enterprise Linux 4 disk image:
><fs> run
><fs> list-filesystems
/dev/vda1: ext3
/dev/VolGroup00/LogVol00: ext3
/dev/VolGroup00/LogVol01: swap
This output shows a Windows disk image:
><fs> run
><fs> list-filesystems
/dev/vda1: ntfs
/dev/vda2: ntfs
Other useful commands are l i st-d evi ces, l i st-parti ti o ns, l vs, pvs, vfs-type and fi l e.
You can get more information and help on any command by typing hel p command, as shown in the following output:
><fs> help vfs-type
NAME
vfs-type - get the Linux VFS type corresponding to a mounted device
SYNOPSIS
vfs-type device
DESCRIPTION
24 5
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
This command gets the filesystem type corresponding to the filesystem on
"device".
For most filesystems, the result is the name of the Linux VFS module
which would be used to mount this filesystem if you mounted it without
specifying the filesystem type. For example a string such as "ext3" or
"ntfs".
To view the actual contents of a file system, it must first be mounted. This example uses one of the
Windows partitions shown in the previous output (/d ev/vd a2), which in this case is known to correspond to the C:\ drive:
><fs> mount-ro /dev/vda2 /
><fs> ll / total 1834753
drwxrwxrwx 1 root root 4096 Nov 1 11:40 .
drwxr-xr-x 21 root root 4096 Nov 16 21:45 ..
lrwxrwxrwx 2 root root 60 Jul 14 2009 Documents and Settings
drwxrwxrwx 1 root root 4096 Nov 15 18:00 Program Files
drwxrwxrwx 1 root root 4096 Sep 19 10:34 Users
drwxrwxrwx 1 root root 16384 Sep 19 10:34 Windows
You can use guestfish commands such as l s, l l , cat, mo re, d o wnl o ad and tar-o ut to view and download files and directories.
Note
There is no concept of a current working directory in this shell. Unlike ordinary shells, you cannot for example use the cd command to change directories. All paths must be fully qualified starting at the top with a forward slash (/) character. Use the Tab key to complete paths.
To exit from the guestfish shell, type exi t or enter C trl + d .
1 7 .4 .1 .2 . Via gue st fish inspe ct io n
Instead of listing and mounting file systems by hand, it is possible to let guestfish itself inspect the image and mount the file systems as they would be in the guest virtual machine. To do this, add the - i option on the command line: guestfish --ro -a /path/to/disk/image -i
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.
Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell
Operating system: Red Hat Enterprise Linux AS release 4 (Nahant Update
24 6
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
8)
/dev/VolGroup00/LogVol00 mounted on /
/dev/vda1 mounted on /boot
><fs> ll /
total 210
drwxr-xr-x. 24 root root 4096 Oct 28 09:09 .
drwxr-xr-x 21 root root 4096 Nov 17 15:10 ..
drwxr-xr-x. 2 root root 4096 Oct 27 22:37 bin
drwxr-xr-x. 4 root root 1024 Oct 27 21:52 boot
drwxr-xr-x. 4 root root 4096 Oct 27 21:21 dev
drwxr-xr-x. 86 root root 12288 Oct 28 09:09 etc
[etc]
Because guestfish needs to start up the libguestfs back end in order to perform the inspection and mounting, the run command is not necessary when using the -i option. The -i option works for many common Linux and Windows guest virtual machines.
1 7 .4 .1 .3. Acce ssing a gue st virt ual m achine by nam e
A guest virtual machine can be accessed from the command line when you specify its name as known to libvirt (in other words, as it appears in vi rsh l i st --al l ). Use the -d option to access a guest virtual machine by its name, with or without the -i option: guestfish --ro -d GuestName -i
17.4 .2. Modifying files wit h guest fish
To modify files, create directories or make other changes to a guest virtual machine, first heed the warning at the beginning of this section: your guest virtual machine must be shut down. Editing or changing a running disk with guestfish will result in disk corruption. This section gives an example of editing the /bo o t/g rub/g rub. co nf file. When you are sure the guest virtual machine is shut down you can omit the --ro flag in order to get write access via a command such as: guestfish -d RHEL3 -i
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.
Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell
Operating system: Red Hat Enterprise Linux AS release 3 (Taroon Update
9)
/dev/vda2 mounted on /
/dev/vda1 mounted on /boot
><fs> edit /boot/grub/grub.conf
Commands to edit files include ed i t, vi and emacs. Many commands also exist for creating files and directories, such as wri te, mkd i r, upl o ad and tar-i n.
17.4 .3. Ot her act ions wit h guest fish
24 7
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
You can also format file systems, create partitions, create and resize LVM logical volumes and much more, with commands such as mkfs, part-ad d , l vresi ze, l vcreate, vg create and pvcreate.
17.4 .4 . Shell script ing wit h guest fish
Once you are familiar with using guestfish interactively, according to your needs, writing shell scripts with it may be useful. The following is a simple shell script to add a new MOTD (message of the day) to a guest:
#!/bin/bash -
set -e
guestname="$1"
guestfish -d "$guestname" -i <<'EOF'
write /etc/motd "Welcome to Acme Incorporated."
chmod 0644 /etc/motd
EOF
17.4 .5. Augeas and libguest fs script ing
Combining libguestfs with Augeas can help when writing scripts to manipulate Linux guest virtual machine configuration. For example, the following script uses Augeas to parse the keyboard configuration of a guest virtual machine, and to print out the layout. Note that this example only works with guest virtual machines running Red Hat Enterprise Linux:
#!/bin/bash -
set -e
guestname="$1"
guestfish -d "$1" -i --ro <<'EOF'
aug-init / 0
aug-get /files/etc/sysconfig/keyboard/LAYOUT
EOF
Augeas can also be used to modify configuration files. You can modify the above script to ch an g e the keyboard layout:
#!/bin/bash -
set -e
guestname="$1"
guestfish -d "$1" -i <<'EOF'
aug-init / 0
aug-set /files/etc/sysconfig/keyboard/LAYOUT '"gb"'
aug-save
EOF
Note the three changes between the two scripts:
1. The --ro option has been removed in the second example, giving the ability to write to the guest virtual machine.
2. The aug -g et command has been changed to aug -set to modify the value instead of fetching it. The new value will be "g b" (including the quotes).
24 8
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
3. The aug -save command is used here so Augeas will write the changes out to disk.
Note
More information about Augeas can be found on the website http://augeas.net
.
guestfish can do much more than we can cover in this introductory document. For example, creating disk images from scratch: guestfish -N fs
Or copying out whole directories from a disk image:
><fs> copy-out /home /tmp/home
For more information see the man page guestfish(1).
17.5. Ot her commands
This section describes tools that are simpler equivalents to using guestfish to view and edit guest virtual machine disk images.
vi rt-cat is similar to the guestfish d o wnl o ad command. It downloads and displays a single file to the guest virtual machine. For example:
# virt-cat RHEL3 /etc/ntp.conf | grep ^server
server 127.127.1.0 # local clock
vi rt-ed i t is similar to the guestfish ed i t command. It can be used to interactively edit a single file within a guest virtual machine. For example, you may need to edit the g rub. co nf file in a
Linux-based guest virtual machine that will not boot:
# virt-edit LinuxGuest /boot/grub/grub.conf
vi rt-ed i t has another mode where it can be used to make simple non-interactive changes to a single file. For this, the - e option is used. This command, for example, changes the root password in a Linux guest virtual machine to having no password:
# virt-edit LinuxGuest /etc/passwd -e 's/^root:.*?:/root::/'
vi rt-l s is similar to the guestfish l s, l l and fi nd commands. It is used to list a directory or directories (recursively). For example, the following command would recursively list files and directories under /home in a Linux guest virtual machine:
# virt-ls -R LinuxGuest /home/ | less
17.6. virt -rescue: T he rescue shell
17.6.1. Int roduct ion
24 9
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
This section describes vi rt-rescue, which can be considered analogous to a rescue CD for virtual machines. It boots a guest virtual machine into a rescue shell so that maintenance can be performed to correct errors and the guest virtual machine can be repaired.
There is some overlap between virt-rescue and guestfish. It is important to distinguish their differing uses. virt-rescue is for making interactive, ad-hoc changes using ordinary Linux file system tools. It is particularly suited to rescuing a guest virtual machine that has failed . virt-rescue cannot be scripted.
In contrast, guestfish is particularly useful for making scripted, structured changes through a formal set of commands (the libguestfs API), although it can also be used interactively.
17.6.2. Running virt -rescue
Before you use vi rt-rescue on a guest virtual machine, make sure the guest virtual machine is not running, otherwise disk corruption will occur. When you are sure the guest virtual machine is not live, enter: virt-rescue GuestName
(where GuestName is the guest name as known to libvirt), or: virt-rescue /path/to/disk/image
(where the path can be any file, any logical volume, LUN, or so on) containing a guest virtual machine disk.
You will first see output scroll past, as virt-rescue boots the rescue VM. In the end you will see:
Welcome to virt-rescue, the libguestfs rescue shell.
Note: The contents of / are the rescue appliance.
You have to mount the guest virtual machine's partitions under /sysroot
before you can examine them.
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
><rescue>
The shell prompt here is an ordinary bash shell, and a reduced set of ordinary Red Hat Enterprise
Linux commands is available. For example, you can enter:
><rescue> fdisk -l /dev/vda
The previous command will list disk partitions. To mount a file system, it is suggested that you mount it under /sysro o t, which is an empty directory in the rescue machine for the user to mount anything you like. Note that the files under / are files from the rescue VM itself:
><rescue> mount /dev/vda1 /sysroot/
EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
><rescue> ls -l /sysroot/grub/
total 324
-rw-r--r--. 1 root root 63 Sep 16 18:14 device.map
-rw-r--r--. 1 root root 13200 Sep 16 18:14 e2fs_stage1_5
250
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
-rw-r--r--. 1 root root 12512 Sep 16 18:14 fat_stage1_5
-rw-r--r--. 1 root root 11744 Sep 16 18:14 ffs_stage1_5
-rw-------. 1 root root 1503 Oct 15 11:19 grub.conf
[...]
When you are finished rescuing the guest virtual machine, exit the shell by entering exi t or C trl + d .
vi rt-rescue has many command line options. The options most often used are:
- - ro : Operate in read-only mode on the guest virtual machine. No changes will be saved. You can use this to experiment with the guest virtual machine. As soon as you exit from the shell, all of your changes are discarded.
- - n et wo rk: Enable network access from the rescue shell. Use this if you need to, for example, download RPM or other files into the guest virtual machine.
17.7. virt -df: Monit oring disk usage
17.7.1. Int roduct ion
This section describes vi rt-d f, which displays file system usage from a disk image or a guest virtual machine. It is similar to the Linux d f command, but for virtual machines.
17.7.2. Running virt -df
To display file system usage for all file systems found in a disk image, enter the following:
# virt-df /dev/vg_guests/RHEL6
Filesystem 1K-blocks Used Available Use%
RHEL6:/dev/sda1 101086 10233 85634 11%
RHEL6:/dev/VolGroup00/LogVol00 7127864 2272744 4493036 32%
(Where /d ev/vg _g uests/R HEL6 is a Red Hat Enterprise Linux 6 guest virtual machine disk image.
The path in this case is the host physical machine logical volume where this disk image is located.)
You can also use vi rt-d f on its own to list information about all of your guest virtual machines (ie.
those known to libvirt). The vi rt-d f command recognizes some of the same options as the standard d f such as -h (human-readable) and -i (show inodes instead of blocks).
vi rt-d f also works on Windows guest virtual machines:
# virt-df -h
Filesystem Size Used Available Use%
F14x64:/dev/sda1 484.2M 66.3M 392.9M 14%
F14x64:/dev/vg_f14x64/lv_root 7.4G 3.0G 4.4G 41%
RHEL6brewx64:/dev/sda1 484.2M 52.6M 406.6M 11%
RHEL6brewx64:/dev/vg_rhel6brewx64/lv_root
13.3G 3.4G 9.2G 26%
Win7x32:/dev/sda1 100.0M 24.1M 75.9M 25%
Win7x32:/dev/sda2 19.9G 7.4G 12.5G 38%
251
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Note
You can use vi rt-d f safely on live guest virtual machines, since it only needs read-only access. However, you should not expect the numbers to be precisely the same as those from a
d f command running inside the guest virtual machine. This is because what is on disk will be slightly out of synch with the state of the live guest virtual machine. Nevertheless it should be a good enough approximation for analysis and monitoring purposes.
virt-df is designed to allow you to integrate the statistics into monitoring tools, databases and so on.
This allows system administrators to generate reports on trends in disk usage, and alerts if a guest virtual machine is about to run out of disk space. To do this you should use the --csv option to generate machine-readable Comma-Separated-Values (CSV) output. CSV output is readable by most databases, spreadsheet software and a variety of other tools and programming languages. The raw
CSV looks like the following:
# virt-df --csv WindowsGuest
Virtual Machine,Filesystem,1K-blocks,Used,Available,Use%
Win7x32,/dev/sda1,102396,24712,77684,24.1%
Win7x32,/dev/sda2,20866940,7786652,13080288,37.3%
For resources and ideas on how to process this output to produce trends and alerts, refer to the following URL: http://virt-tools.org/learning/advanced-virt-df/ .
17.8. virt -resize: resizing guest virt ual machines offline
17.8.1. Int roduct ion
This section describes vi rt-resi ze, a tool for expanding or shrinking guest virtual machines. It only works for guest virtual machines which are offline (shut down). It works by copying the guest virtual machine image and leaving the original disk image untouched. This is ideal because you can use the original image as a backup, however there is a trade-off as you need twice the amount of disk space.
17.8.2. Expanding a disk image
This section demonstrates a simple case of expanding a disk image:
1. Locate the disk image to be resized. You can use the command vi rsh d umpxml
G uestName for a libvirt guest virtual machine.
2. Decide on how you wish to expand the guest virtual machine. Run vi rt-d f -h and vi rt-
l i st-parti ti o ns -l h on the guest virtual machine disk, as shown in the following output:
# virt-df -h /dev/vg_guests/RHEL6
Filesystem Size Used Available Use%
RHEL6:/dev/sda1 98.7M 10.0M 83.6M 11%
RHEL6:/dev/VolGroup00/LogVol00 6.8G 2.2G 4.3G 32%
# virt-list-partitions -lh /dev/vg_guests/RHEL6
/dev/sda1 ext3 101.9M
/dev/sda2 pv 7.9G
252
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
This example will demonstrate how to:
Increase the size of the first (boot) partition, from approximately 100MB to 500MB.
Increase the total disk size from 8GB to 16GB.
Expand the second partition to fill the remaining space.
Expand /d ev/Vo l G ro up0 0 /Lo g Vo l 0 0 to fill the new space in the second partition.
1. Make sure the guest virtual machine is shut down.
2. Rename the original disk as the backup. How you do this depends on the host physical machine storage environment for the original disk. If it is stored as a file, use the mv command. For logical volumes (as demonstrated in this example), use l vrename:
# lvrename /dev/vg_guests/RHEL6 /dev/vg_guests/RHEL6.backup
3. Create the new disk. The requirements in this example are to expand the total disk size up to
16GB. Since logical volumes are used here, the following command is used:
# lvcreate -L 16G -n RHEL6 /dev/vg_guests
Logical volume "RHEL6" created
4. The requirements from step 2 are expressed by this command:
# virt-resize \
/dev/vg_guests/RHEL6.backup /dev/vg_guests/RHEL6 \
--resize /dev/sda1=500M \
--expand /dev/sda2 \
--LV-expand /dev/VolGroup00/LogVol00
The first two arguments are the input disk and output disk. --resi ze /d ev/sd a1= 50 0 M resizes the first partition up to 500MB. --expand /d ev/sd a2 expands the second partition to fill all remaining space. --LV-expand /d ev/Vo l G ro up0 0 /Lo g Vo l 0 0 expands the guest virtual machine logical volume to fill the extra space in the second partition.
vi rt-resi ze describes what it is doing in the output:
Summary of changes:
/dev/sda1: partition will be resized from 101.9M to 500.0M
/dev/sda1: content will be expanded using the 'resize2fs' method
/dev/sda2: partition will be resized from 7.9G to 15.5G
/dev/sda2: content will be expanded using the 'pvresize' method
/dev/VolGroup00/LogVol00: LV will be expanded to maximum size
/dev/VolGroup00/LogVol00: content will be expanded using the
'resize2fs' method
Copying /dev/sda1 ...
[#####################################################]
Copying /dev/sda2 ...
[#####################################################]
Expanding /dev/sda1 using the 'resize2fs' method
Expanding /dev/sda2 using the 'pvresize' method
Expanding /dev/VolGroup00/LogVol00 using the 'resize2fs' method
253
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
5. Try to boot the virtual machine. If it works (and after testing it thoroughly) you can delete the backup disk. If it fails, shut down the virtual machine, delete the new disk, and rename the backup disk back to its original name.
6. Use vi rt-d f and/or vi rt-l i st-parti ti o ns to show the new size:
# virt-df -h /dev/vg_pin/RHEL6
Filesystem Size Used Available Use%
RHEL6:/dev/sda1 484.4M 10.8M 448.6M 3%
RHEL6:/dev/VolGroup00/LogVol00 14.3G 2.2G 11.4G 16%
Resizing guest virtual machines is not an exact science. If vi rt-resi ze fails, there are a number of tips that you can review and attempt in the virt-resize(1) man page. For some older Red Hat Enterprise
Linux guest virtual machines, you may need to pay particular attention to the tip regarding GRUB.
17.9. virt -inspect or: inspect ing guest virt ual machines
17.9.1. Int roduct ion
vi rt-i nspecto r is a tool for inspecting a disk image to find out what operating system it contains.
Note
Red Hat Enterprise Linux 6.2 ships with two variations of this program: vi rt-i nspecto r is the original program as found in Red Hat Enterprise Linux 6.0 and is now deprecated upstream. vi rt-i nspecto r2 is the same as the new upstream vi rt-i nspecto r program.
17.9.2. Inst allat ion
To install virt-inspector and the documentation, enter the following command:
# yum install libguestfs-tools libguestfs-devel
To process Windows guest virtual machines you must also install libguestfs-winsupport. Refer to
Section 17.10.2, “ Installation”
for details. The documentation, including example XML output and a
Relax-NG schema for the output, will be installed in /usr/share/d o c/l i bg uestfs-d evel -*/ where "*" is replaced by the version number of libguestfs.
17.9.3. Running virt -inspect or
You can run vi rt-i nspecto r against any disk image or libvirt guest virtual machine as shown in the following example: virt-inspector --xml disk.img > report.xml
Or as shown here: virt-inspector --xml GuestName > report.xml
254
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
The result will be an XML report (repo rt. xml ). The main components of the XML file are a top-level
<operatingsytems> element containing usually a single <operatingsystem> element, similar to the following:
<operatingsystems>
<operatingsystem>
<!-- the type of operating system and Linux distribution -->
<name>linux</name>
<distro>rhel</distro>
<!-- the name, version and architecture -->
<product_name>Red Hat Enterprise Linux Server release 6.4
</product_name>
<major_version>6</major_version>
<minor_version>4</minor_version>
<package_format>rpm</package_format>
<package_management>yum</package_management>
<root>/dev/VolGroup/lv_root</root>
<!-- how the filesystems would be mounted when live -->
<mountpoints>
<mountpoint dev="/dev/VolGroup/lv_root">/</mountpoint>
<mountpoint dev="/dev/sda1">/boot</mountpoint>
<mountpoint dev="/dev/VolGroup/lv_swap">swap</mountpoint>
</mountpoints>
< !-- filesystems-->
<filesystem dev="/dev/VolGroup/lv_root">
<label></label>
<uuid>b24d9161-5613-4ab8-8649-f27a8a8068d3</uuid>
<type>ext4</type>
<content>linux-root</content>
<spec>/dev/mapper/VolGroup-lv_root</spec>
</filesystem>
<filesystem dev="/dev/VolGroup/lv_swap">
<type>swap</type>
<spec>/dev/mapper/VolGroup-lv_swap</spec>
</filesystem>
<!-- packages installed -->
<applications>
<application>
<name>firefox</name>
<version>3.5.5</version>
<release>1.fc12</release>
</application>
</applications>
</operatingsystem>
</operatingsystems>
Processing these reports is best done using W3C standard XPath queries. Red Hat Enterprise Linux
6 comes with a command line program (xpath) which can be used for simple instances; however, for long-term and advanced usage, you should consider using an XPath library along with your favorite programming language.
As an example, you can list out all file system devices using the following XPath query:
255
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
virt-inspector --xml GuestName | xpath //filesystem/@ dev
Found 3 nodes:
-- NODE --
dev="/dev/sda1"
-- NODE --
dev="/dev/vg_f12x64/lv_root"
-- NODE --
dev="/dev/vg_f12x64/lv_swap"
Or list the names of all applications installed by entering:
virt-inspector --xml GuestName | xpath //application/name
[...long list...]
17.10. virt -win-reg: Reading and edit ing t he Windows Regist ry
17.10.1. Int roduct ion
vi rt-wi n-reg is a tool that manipulates the Registry in Windows guest virtual machines. It can be used to read out registry keys. You can also use it to make changes to the Registry, but you must
n ever try to do this for live/running guest virtual machines, as it will result in disk corruption.
17.10.2. Inst allat ion
To use vi rt-wi n-reg you must run the following:
# yum install libguestfs-tools libguestfs-winsupport
17.10.3. Using virt -win-reg
To read out Registry keys, specify the name of the guest virtual machine (or its disk image) and the name of the Registry key. You must use single quotes to surround the name of the desired key:
# virt-win-reg WindowsGuest \
'HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Uninstall'
\
| less
The output is in the standard text-based format used by . R EG files on Windows.
256
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
Note
Hex-quoting is used for strings because the format does not properly define a portable encoding method for strings. This is the only way to ensure fidelity when transporting . R EG files from one machine to another.
You can make hex-quoted strings printable by piping the output of vi rt-wi n-reg through this simple Perl script: perl -MEncode -pe's?hex\((\d+)\):(\S+)?
$t=$1;$_=$2;s,\,,,g;"str($t):\"".decode(utf16le=>pack("H*",$_))."\""
?eg'
To merge changes into the Windows Registry of an offline guest virtual machine, you must first prepare a . R EG file. There is a great deal of documentation about doing this available here . When you have prepared a . R EG file, enter the following:
# virt-win-reg --merge WindowsGuest input.reg
This will update the registry in the guest virtual machine.
17.11. Using t he API from Programming Languages
The libguestfs API can be used directly from the following languages in Red Hat Enterprise Linux 6.2:
C, C++, Perl, Python, Java, Ruby and OCaml.
To install C and C++ bindings, enter the following command:
# yum install libguestfs-devel
To install Perl bindings:
# yum install 'perl(Sys::Guestfs)'
To install Python bindings:
# yum install python-libguestfs
To install Java bindings:
# yum install libguestfs-java libguestfs-java-devel libguestfs-javadoc
To install Ruby bindings:
# yum install ruby-libguestfs
To install OCaml bindings:
# yum install ocaml-libguestfs ocaml-libguestfs-devel
257
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The binding for each language is essentially the same, but with minor syntactic changes. A C statement: guestfs_launch (g);
Would appear like the following in Perl:
$g->launch ()
Or like the following in OCaml: g#launch ()
Only the API from C is detailed in this section.
In the C and C++ bindings, you must manually check for errors. In the other bindings, errors are converted into exceptions; the additional error checks shown in the examples below are not necessary for other languages, but conversely you may wish to add code to catch exceptions. Refer to the following list for some points of interest regarding the architecture of the libguestfs API:
The libguestfs API is synchronous. Each call blocks until it has completed. If you want to make calls asynchronously, you have to create a thread.
The libguestfs API is not thread safe: each handle should be used only from a single thread, or if you want to share a handle between threads you should implement your own mutex to ensure that two threads cannot execute commands on one handle at the same time.
You should not open multiple handles on the same disk image. It is permissible if all the handles are read-only, but still not recommended.
You should not add a disk image for writing if anything else could be using that disk image (eg. a live VM). Doing this will cause disk corruption.
Opening a read-only handle on a disk image which is currently in use (eg. by a live VM) is possible; however, the results may be unpredictable or inconsistent particularly if the disk image is being heavily written to at the time you are reading it.
17.11.1. Int eract ion wit h t he API via a C program
Your C program should start by including the <guestfs.h> header file, and creating a handle:
#include <stdio.h>
#include <stdlib.h>
#include <guestfs.h> int main (int argc, char *argv[])
{
guestfs_h *g;
g = guestfs_create ();
if (g == NULL) {
perror ("failed to create libguestfs handle");
exit (EXIT_FAILURE);
}
258
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
/* ... */
guestfs_close (g);
exit (EXIT_SUCCESS);
}
Save this program to a file (test. c). Compile this program and run it with the following two commands: gcc -Wall test.c -o test -lguestfs
./test
At this stage it should print no output. The rest of this section demonstrates an example showing how to extend this program to create a new disk image, partition it, format it with an ext4 file system, and create some files in the file system. The disk image will be called d i sk. i mg and be created in the current directory.
The outline of the program is:
Create the handle.
Add disk(s) to the handle.
Launch the libguestfs back end.
Create the partition, file system and files.
Close the handle and exit.
Here is the modified program:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <guestfs.h>
int
main (int argc, char *argv[])
{
guestfs_h *g;
size_t i;
g = guestfs_create ();
if (g == NULL) {
perror ("failed to create libguestfs handle");
exit (EXIT_FAILURE);
}
/* Create a raw-format sparse disk image, 512 MB in size. */
int fd = open ("disk.img", O_CREAT|O_WRONLY|O_TRUNC|O_NOCTTY, 0666);
if (fd == -1) {
perror ("disk.img");
exit (EXIT_FAILURE);
}
259
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
if (ftruncate (fd, 512 * 1024 * 1024) == -1) {
perror ("disk.img: truncate");
exit (EXIT_FAILURE);
}
if (close (fd) == -1) {
perror ("disk.img: close");
exit (EXIT_FAILURE);
}
/* Set the trace flag so that we can see each libguestfs call. */
guestfs_set_trace (g, 1);
/* Set the autosync flag so that the disk will be synchronized
* automatically when the libguestfs handle is closed.
*/
guestfs_set_autosync (g, 1);
/* Add the disk image to libguestfs. */
if (guestfs_add_drive_opts (g, "disk.img",
GUESTFS_ADD_DRIVE_OPTS_FORMAT, "raw", /* raw format */
GUESTFS_ADD_DRIVE_OPTS_READONLY, 0, /* for write */
-1 /* this marks end of optional arguments */ )
== -1)
exit (EXIT_FAILURE);
/* Run the libguestfs back-end. */
if (guestfs_launch (g) == -1)
exit (EXIT_FAILURE);
/* Get the list of devices. Because we only added one drive
* above, we expect that this list should contain a single
* element.
*/
char **devices = guestfs_list_devices (g);
if (devices == NULL)
exit (EXIT_FAILURE);
if (devices[0] == NULL || devices[1] != NULL) {
fprintf (stderr,
"error: expected a single device from list-devices\n");
exit (EXIT_FAILURE);
}
/* Partition the disk as one single MBR partition. */
if (guestfs_part_disk (g, devices[0], "mbr") == -1)
exit (EXIT_FAILURE);
/* Get the list of partitions. We expect a single element, which
* is the partition we have just created.
*/
char **partitions = guestfs_list_partitions (g);
if (partitions == NULL)
exit (EXIT_FAILURE);
if (partitions[0] == NULL || partitions[1] != NULL) {
fprintf (stderr,
"error: expected a single partition from listpartitions\n");
260
Chapt er 1 7 . G uest virt ual machine disk access wit h offline t ools
exit (EXIT_FAILURE);
}
/* Create an ext4 filesystem on the partition. */
if (guestfs_mkfs (g, "ext4", partitions[0]) == -1)
exit (EXIT_FAILURE);
/* Now mount the filesystem so that we can add files. */
if (guestfs_mount_options (g, "", partitions[0], "/") == -1)
exit (EXIT_FAILURE);
/* Create some files and directories. */
if (guestfs_touch (g, "/empty") == -1)
exit (EXIT_FAILURE);
const char *message = "Hello, world\n";
if (guestfs_write (g, "/hello", message, strlen (message)) == -1)
exit (EXIT_FAILURE);
if (guestfs_mkdir (g, "/foo") == -1)
exit (EXIT_FAILURE);
/* This uploads the local file /etc/resolv.conf into the disk image.
*/
if (guestfs_upload (g, "/etc/resolv.conf", "/foo/resolv.conf") == -1)
exit (EXIT_FAILURE);
/* Because 'autosync' was set (above) we can just close the handle
* and the disk contents will be synchronized. You can also do
* this manually by calling guestfs_umount_all and guestfs_sync.
*/
guestfs_close (g);
/* Free up the lists. */
for (i = 0; devices[i] != NULL; ++i)
free (devices[i]);
free (devices);
for (i = 0; partitions[i] != NULL; ++i)
free (partitions[i]);
free (partitions);
exit (EXIT_SUCCESS);
}
Compile and run this program with the following two commands: gcc -Wall test.c -o test -lguestfs
./test
If the program runs to completion successfully then you should be left with a disk image called
d i sk. i mg , which you can examine with guestfish: guestfish --ro -a disk.img -m /dev/sda1
><fs> ll /
><fs> cat /foo/resolv.conf
261
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
By default (for C and C++ bindings only), libguestfs prints errors to stderr. You can change this behavior by setting an error handler. The guestfs(3) man page discusses this in detail.
17.12. T roubleshoot ing
A test tool is available to check that libguestfs is working. Run the following command after installing libguestfs (root access not required) to test for normal operation:
$ libguestfs-test-tool
This tool prints a large amount of text to test the operation of libguestfs. If the test is successful, the following text will appear near the end of the output:
===== TEST FINISHED OK =====
17.13. Where t o find furt her document at ion
The primary source for documentation for libguestfs and the tools are the Unix man pages. The API is documented in guestfs(3). guestfish is documented in guestfish(1). The virt tools are documented in their own man pages (eg. virt-df(1)).
262
Chapt er 1 8 . Using simple t ools for guest virt ual machine management
Chapter 18. Using simple tools for guest virtual machine management
In addition to virt-manager, there are other smaller more minimal tools that can allow you to have access to your guest virtual machine's console. The sections that follow describe and explain these tools.
18.1. Using virt -viewer
virt-viewer is a minimal tool for displaying the graphical console of a guest virtual machine. The console is accessed using the VNC or SPICE protocol. The guest can be referred to based on its name, ID, or UUID. If the guest is not already running, then the viewer can be told to wait until is starts before attempting to connect to the console. The viewer can connect to remote hosts to lookup the console information and then also connect to the remote console using the same network transport.
To install the virt-viewer tool, run:
# sud o yum i nstal l vi rt-vi ewer
The basic virt viewer commands are as follows:
# vi rt-vi ewer [O P T IO NS] D O MAIN-NAME| ID | UUID
The following options may be used with virt-viewer
-h, or --hel p - Displays the command line help summary.
-V, or --versi o n - Displays the virt-viewer version number.
-v, or --verbo se - Displays information about the connection to the guest virtual machine
-c [UR I], or --co nnect= UR I - Specifies the hypervisor connection URI
-w, or --wai t - Causes the domain to start up before attempting to connect to the console.
-r, or --reco nnect - Automatically reconnects to the domain if it shuts down and restarts
-z P C T , or --zo o m= P C T - Adjusts the zoom level of the display window in the specified percentage. Accepted range 10-200% .
-a, or --attach - Uses libvirt to directly attach to a local display, instead of making a TCP/UNIX socket connection. This avoids the need to authenticate with the remote display, if authentication with libvirt is already allowed. This option does not work with remote displays.
-f, or --ful l -screen - Starts with the command window maximized to its fullscreen size.
-h HO T KEY S, or --ho tkeys HO T KEY S - Overrides the default hotkey settings with the new
specified hotkey. Refer to Example 18.4, “ Setting hot keys”
.
--d ebug - Prints debugging information
Examp le 18.1. Co n n ect in g t o a g u est virt u al mach in e
If using a XEN hypervisor:
263
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
# vi rt-vi ewer g uest-name
If using a KVM-QEMU hypervisor:
# vi rt-vi ewer --co nnect q emu: ///system 7
Examp le 18.2. T o wait f o r a sp ef ic g u est t o st art b ef o re co n n ect in g
Enter the following command:
# vi rt-vi ewer --reco nnect --wai t 6 6 ab33c0 -6 9 19 -a3f7-e6 59 -16 c82d 24 8521
Examp le 18.3. T o co n n ect t o a remo t e co n so le u sin g T LS
Enter the following command:
#vi rt-vi ewer --co nnect xen: //exampl e. o rg / d emo
To connect to a remote host using SSH, lookup the guest configuration and then make a direct non-tunnelled connection to the console
Examp le 18.4 . Set t in g h o t keys
To create a customized hotkey, run the following command:
# virt-viewer --hotkeys=([action1]=[key-combination1]), ([action2]=[keycombination2])
The following actions can be assigned to a hotkey: toggle-fullscreen release-cursor smartcard-insert smartcard-remove
Key name combinations are case insensitive. Each hot key setting should have a unique key combination.
For example, to create a hotkey to change to full screen mode:
#vi rt-vi ewer --ho tkeys= to g g l e-ful l screen= shi ft+ f11 q emu: ///system 7
18.2. remot e-viewer
The remote-viewer is a simple remote desktop display client that supports SPICE and VNC.
To install the virt-viewer tool, run:
264
Chapt er 1 8 . Using simple t ools for guest virt ual machine management
# sud o yum i nstal l remo te-vi ewer
The basic remote viewer commands are as follows:remo te-vi ewer [O P T IO NS] D O MAIN-
NAME| ID | UUID
The following options may be used with remote-viewer
-h, or --hel p - Displays the command line help summary.
-V, or --versi o n - Displays the remote-viewer version number.
-v, or --verbo se - Displays information about the connection to the guest virtual machine
-z P C T , or --zo o m= P C T - Adjusts the zoom level of the display window in the specified percentage. Accepted range 10-200% .
-f, or --ful l -screen - Starts with the command window maximized to its fullscreen size.
-t T IT LE, or --ti tl e T IT LE - Sets the window title to the string given
--spi ce-co ntro l l er - Uses the SPICE controller to initialize the connection with the SPICE server. This option is used by the SPICE browser addons to allow web page to start a client.
--d ebug - Prints debugging information
For more information see the MAN page for the remote-viewer.
265
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Chapter 19. Virtual Networking
This chapter introduces the concepts needed to create, start, stop, remove, and modify virtual networks with libvirt.
Additional information can be found in the libvirt reference chapter
19.1. Virt ual net work swit ches
Libvirt virtual networking uses the concept of a virtual network switch. A virtual network switch is a software construct that operates on a host physical machine server, to which virtual machines
(guests) connect. The network traffic for a guest is directed through this switch:
Fig u re 19 .1. Virt u al n et wo rk swit ch wit h t wo g u est s
Linux host physical machine servers represent a virtual network switch as a network interface. When the libvirtd daemon (l i bvi rtd ) is first installed and started, the default network interface representing the virtual network switch is vi rbr0 .
266
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .2. Lin u x h o st p h ysical mach in e wit h an in t erf ace t o a virt u al n et wo rk swit ch
This vi rbr0 interface can be viewed with the i fco nfi g and i p commands like any other interface:
$ ifconfig virbr0
virbr0 Link encap:Ethernet HWaddr 1B:C4:94:CF:FD:17
inet addr:192.168.122.1 Bcast:192.168.122.255
Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:3097 (3.0 KiB)
$ ip addr show virbr0
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 1b:c4:94:cf:fd:17 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
19.2. Net work Address T ranslat ion
By default, virtual network switches operate in NAT mode. They use IP masquerading rather than
SNAT (Source-NAT) or DNAT (Destination-NAT). IP masquerading enables connected guests to use the host physical machine IP address for communication to any external network. By default, computers that are placed externally to the host physical machine cannot communicate to the guests inside when the virtual network switch is operating in NAT mode, as shown in the following diagram:
267
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 19 .3. Virt u al n et wo rk swit ch u sin g NAT wit h t wo g u est s
Warning
Virtual network switches use NAT configured by iptables rules. Editing these rules while the switch is running is not recommended, as incorrect rules may result in the switch being unable to communicate.
If the switch is not running, you can set th public IP range for forward mode NAT in order to create a port masquerading range by running:
# i ptabl es -j SNAT --to -so urce [start]-[end ]
19.3. Net working prot ocols
The following sections describe individual networking protocols and how they are used in libvirt
19.3.1. DNS and DHCP
IP information can be assigned to guests via DHCP. A pool of addresses can be assigned to a virtual network switch for this purpose. Libvirt uses the d nsmasq program for this. An instance of dnsmasq is automatically configured and started by libvirt for each virtual network switch that needs it.
268
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .4 . Virt u al n et wo rk swit ch ru n n in g d n smasq
19.3.2. Rout ed mode
When using routed mode, the virtual switch connects to the physical LAN connected to the host physical machine, passing traffic back and forth without the use of NAT. The virtual switch can examine all traffic and use the information contained within the network packets to make routing decisions. When using this mode, all of the virtual machines are in their own subnet, routed through a virtual switch. This situation is not always ideal as no other host physical machines on the physical network are aware of the virtual machines without manual physical router configuration, and cannot access the virtual machines. Routed mode operates at Layer 3 of the OSI networking model.
269
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 19 .5. Virt u al n et wo rk swit ch in ro u t ed mo d e
19.3.3. Isolat ed mode
When using Isolated mode, guests connected to the virtual switch can communicate with each other, and with the host physical machine, but their traffic will not pass outside of the host physical machine, nor can they receive traffic from outside the host physical machine. Using dnsmasq in this mode is required for basic functionality such as DHCP. However, even if this network is isolated from any physical network, DNS names are still resolved. Therefore a situation can arise when DNS names resolve but ICMP echo request (ping) commands fail.
270
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .6 . Virt u al n et wo rk swit ch in iso lat ed mo d e
19.4 . T he default configurat ion
When the libvirtd daemon (l i bvi rtd ) is first installed, it contains an initial virtual network switch configuration in NAT mode. This configuration is used so that installed guests can communicate to the external network, through the host physical machine. The following image demonstrates this default configuration for l i bvi rtd :
Fig u re 19 .7. Def au lt lib virt n et wo rk co n f ig u rat io n
Note
A virtual network can be restricted to a specific physical interface. This may be useful on a physical system that has several interfaces (for example, eth0 , eth1 and eth2). This is only useful in routed and NAT modes, and can be defined in the d ev= <i nterface> option, or in
vi rt-manag er when creating a new virtual network.
19.5. Examples of common scenarios
This section demonstrates different virtual networking modes and provides some example scenarios.
271
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
19.5.1. Rout ed mode
DMZ
Consider a network where one or more nodes are placed in a controlled subnetwork for security reasons. The deployment of a special subnetwork such as this is a common practice, and the subnetwork is known as a DMZ . Refer to the following diagram for more details on this layout:
Fig u re 19 .8. Samp le DMZ co n f ig u rat io n
Host physical machines in a DMZ typically provide services to WAN (external) host physical machines as well as LAN (internal) host physical machines. As this requires them to be accessible from multiple locations, and considering that these locations are controlled and operated in different ways based on their security and trust level, routed mode is the best configuration for this environment.
Virt u al Server h o st in g
Consider a virtual server hosting company that has several host physical machines, each with two physical network connections. One interface is used for management and accounting, the other is for the virtual machines to connect through. Each guest has its own public IP address, but the host physical machines use private IP address as management of the guests can only be performed by internal administrators. Refer to the following diagram to understand this scenario:
272
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .9 . Virt u al server h o st in g samp le co n f ig u rat io n
When the host physical machine has a public IP address and the virtual machines have static public
IP addresses, bridged networking cannot be used, as the provider only accepts packets from the
MAC address of the public host physical machine. The following diagram demonstrates this:
Fig u re 19 .10. Virt u al server u sin g st at ic IP ad d resses
19.5.2. NAT mode
NAT (Network Address Translation) mode is the default mode. It can be used for testing when there is no need for direct network visibility.
19.5.3. Isolat ed mode
273
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Isolated mode allows virtual machines to communicate with each other only. They are unable to interact with the physical network.
19.6. Managing a virt ual net work
To configure a virtual network on your system:
1. From the Ed i t menu, select C o nnecti o n D etai l s.
Fig u re 19 .11. Select in g a h o st p h ysical mach in e' s d et ails
2. This will open the Co n n ect io n Det ails menu. Click the Vi rtual Netwo rks tab.
274
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .12. Virt u al n et wo rk co n f ig u rat io n
3. All available virtual networks are listed on the left-hand box of the menu. You can edit the configuration of a virtual network by selecting it from this box and editing as you see fit.
19.7. Creat ing a virt ual net work
To create a virtual network on your system:
1. Open the Vi rtual Netwo rks tab from within the C o nnecti o n D etai l s menu. Click the
Ad d Netwo rk button, identified by a plus sign (+) icon. For more information, refer to
Section 19.6, “ Managing a virtual network” .
Fig u re 19 .13. Virt u al n et wo rk co n f ig u rat io n
This will open the Creat e a n ew virt u al n et wo rk window. Click Fo rward to continue.
275
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
276
Fig u re 19 .14 . Creat in g a n ew virt u al n et wo rk
2. Enter an appropriate name for your virtual network and click Fo rward .
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .15. Namin g yo u r virt u al n et wo rk
3. Enter an IPv4 address space for your virtual network and click Fo rward .
277
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
278
Fig u re 19 .16 . Ch o o sin g an IPv4 ad d ress sp ace
4. Define the DHCP range for your virtual network by specifying a Start and End range of IP addresses. Click Fo rward to continue.
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .17. Select in g t h e DHCP ran g e
5. Select how the virtual network should connect to the physical network.
279
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
280
Fig u re 19 .18. Co n n ect in g t o p h ysical n et wo rk
If you select Fo rward i ng to physi cal netwo rk, choose whether the D esti nati o n should be Any physi cal d evi ce or a specific physical device. Also select whether the
Mo d e should be NAT or R o uted .
Click Fo rward to continue.
6. You are now ready to create the network. Check the configuration of your network and click
Fi ni sh.
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .19 . Read y t o creat e n et wo rk
7. The new virtual network is now available in the Vi rtual Netwo rks tab of the C o nnecti o n
D etai l s window.
19.8. At t aching a virt ual net work t o a guest
To attach a virtual network to a guest:
1. In the Vi rtual Machi ne Manag er window, highlight the guest that will have the network assigned.
281
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 19 .20. Select in g a virt u al mach in e t o d isp lay
2. From the Virtual Machine Manager Ed i t menu, select Vi rtual Machi ne D etai l s.
282
Fig u re 19 .21. Disp layin g t h e virt u al mach in e d et ails
3. Click the Ad d Hard ware button on the Virtual Machine Details window.
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .22. T h e Virt u al Mach in e Det ails win d o w
4. In the Ad d new vi rtual hard ware window, select Netwo rk from the left pane, and select your network name (network1 in this example) from the Ho st d evi ce menu and click
Fi ni sh.
283
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 19 .23. Select yo u r n et wo rk f ro m t h e Ad d n ew virt u al h ard ware win d o w
5. The new network is now displayed as a virtual network interface that will be presented to the guest upon launch.
284
Chapt er 1 9 . Virt ual Net working
Fig u re 19 .24 . New n et wo rk sh o wn in g u est h ard ware list
19.9. Direct ly at t aching t o physical int erface
The instructions provided in this chapter will assist in the direct attachment of the virtual machine's
NIC to the given physical interface of the host physical machine. This setup requires the Linux macvtap driver to be available. There are four modes that you can choose for the operation mode of the macvtap device, with 'vepa' being the default mode. Their behavior is as follows:
Ph ysical in t erf ace d elivery mo d es vep a
All VMs' packets are sent to the external bridge. Packets whose destination is a VM on the same host physical machine as where the packet originates from are sent back to the host physical machine by the VEPA capable bridge (today's bridges are typically not VEPA capable).
b rid g e
Packets whose destination is on the same host physical machine as where they originate from are directly delivered to the target macvtap device. Both origin and destination devices need to be in bridge mode for direct delivery. If either one of them is in vepa mode, a VEPA capable bridge is required.
285
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide p rivat e
All packets are sent to the external bridge and will only be delivered to a target VM on the same host physical machine if they are sent through an external router or gateway and that device sends them back to the host physical machine. This procedure is followed if either the source or destination device is in private mode.
p asst h ro u g h
This feature attaches a virtual function of a SRIOV capable NIC directly to a VM without losing the migration capability. All packets are sent to the VF/IF of the configured network device. Depending on the capabilities of the device additional prerequisites or limitations may apply; for example, on Linux this requires kernel 2.6.38 or newer.
Each of the four modes is configured by changing the domain xml file. Once this file is opened, change the mode setting as shown:
<devices>
...
<interface type='direct'>
<source dev='eth0' mode='vepa'/>
</interface>
</devices>
The network access of direct attached guest virtual machines can be managed by the hardware switch to which the physical interface of the host physical machine is connected to.
The interface can have additional parameters as shown below, if the switch is conforming to the IEEE
802.1Qbg standard. The parameters of the virtualport element are documented in more detail in the
IEEE 802.1Qbg standard. The values are network specific and should be provided by the network administrator. In 802.1Qbg terms, the Virtual Station Interface (VSI) represents the virtual interface of a virtual machine.
Note that IEEE 802.1Qbg requires a non-zero value for the VLAN ID. Also if the switch is conforming to the IEEE 802.1Qbh standard, the values are network specific and should be provided by the network administrator.
Virt u al St at io n In t erf ace t yp es man ag erid
The VSI Manager ID identifies the database containing the VSI type and instance definitions. This is an integer value and the value 0 is reserved.
t yp eid
The VSI Type ID identifies a VSI type characterizing the network access. VSI types are typically managed by network administrator. This is an integer value.
t yp eid versio n
The VSI Type Version allows multiple versions of a VSI Type. This is an integer value.
in st an ceid
The VSI Instance ID Identifier is generated when a VSI instance (i.e. a virtual interface of a virtual machine) is created. This is a globally unique identifier.
p ro f ileid
286
Chapt er 1 9 . Virt ual Net working
The profile ID contains the name of the port profile that is to be applied onto this interface.
This name is resolved by the port profile database into the network parameters from the port profile, and those network parameters will be applied to this interface.
Each of the four types is configured by changing the domain xml file. Once this file is opened, change the mode setting as shown:
<devices>
...
<interface type='direct'>
<source dev='eth0.2' mode='vepa'/>
<virtualport type="802.1Qbg">
<parameters managerid="11" typeid="1193047" typeidversion="2" instanceid="09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f"/>
</virtualport>
</interface>
</devices>
The profile ID is shown here:
<devices>
...
<interface type='direct'>
<source dev='eth0' mode='private'/>
<virtualport type='802.1Qbh'>
<parameters profileid='finance'/>
</virtualport>
</interface>
</devices>
...
19.10. Applying net work filt ering
This section provides an introduction to libvirt's network filters, their goals, concepts and XML format.
19.10.1. Int roduct ion
The goal of the network filtering, is to enable administrators of a virtualized system to configure and enforce network traffic filtering rules on virtual machines and manage the parameters of network traffic that virtual machines are allowed to send or receive. The network traffic filtering rules are applied on the host physical machine when a virtual machine is started. Since the filtering rules cannot be circumvented from within the virtual machine, it makes them mandatory from the point of view of a virtual machine user.
From the point of view of the guest virtual machine, the network filtering system allows each virtual machine's network traffic filtering rules to be configured individually on a per interface basis. These rules are applied on the host physical machine when the virtual machine is started and can be modified while the virtual machine is running. The latter can be achieved by modifying the XML description of a network filter.
Multiple virtual machines can make use of the same generic network filter. When such a filter is modified, the network traffic filtering rules of all running virtual machines that reference this filter are updated. The machines that are not running will update on start.
287
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
As previously mentioned, applying network traffic filtering rules can be done on individual network interfaces that are configured for certain types of network configurations. Supported network types include: network ethernet -- must be used in bridging mode bridge
Examp le 19 .1. An examp le o f n et wo rk f ilt erin g
The interface XML is used to reference a top-level filter. In the following example, the interface description references the filter clean-traffic.
<devices>
<interface type='bridge'>
<mac address='00:16:3e:5d:c7:9e'/>
<filterref filter='clean-traffic'/>
</interface>
</devices>
Network filters are written in XML and may either contain: references to other filters, rules for traffic filtering, or hold a combination of both. The above referenced filter clean-traffic is a filter that only contains references to other filters and no actual filtering rules. Since references to other filters can be used, a tree of filters can be built. The clean-traffic filter can be viewed using the command: #
vi rsh nwfi l ter-d umpxml cl ean-traffi c.
As previously mentioned, a single network filter can be referenced by multiple virtual machines.
Since interfaces will typically have individual parameters associated with their respective traffic filtering rules, the rules described in a filter's XML can be generalized using variables. In this case, the variable name is used in the filter XML and the name and value are provided at the place where the filter is referenced.
Examp le 19 .2. Descrip t io n ext en d ed
In the following example, the interface description has been extended with the parameter IP and a dotted IP address as a value.
<devices>
<interface type='bridge'>
<mac address='00:16:3e:5d:c7:9e'/>
<filterref filter='clean-traffic'>
<parameter name='IP' value='10.0.0.1'/>
</filterref>
</interface>
</devices>
In this particular example, the clean-traffic network traffic filter will be represented with the IP address parameter 10.0.0.1 and as per the rule dictates that all traffic from this interface will always be using 10.0.0.1 as the source IP address, which is one of the purpose of this particular filter.
19.10.2. Filt ering chains
288
Chapt er 1 9 . Virt ual Net working
Filtering rules are organized in filter chains. These chains can be thought of as having a tree structure with packet filtering rules as entries in individual chains (branches).
Packets start their filter evaluation in the root chain and can then continue their evaluation in other chains, return from those chains back into the root chain or be dropped or accepted by a filtering rule in one of the traversed chains.
Libvirt's network filtering system automatically creates individual root chains for every virtual machine's network interface on which the user chooses to activate traffic filtering. The user may write filtering rules that are either directly instantiated in the root chain or may create protocol-specific filtering chains for efficient evaluation of protocol-specific rules.
The following chains exist: root mac stp (spanning tree protocol) vlan arp and rarp ipv4 ipv6
Multiple chains evaluating the mac, stp, vlan, arp, rarp, ipv4, or ipv6 protocol can be created using the protocol name only as a prefix in the chain's name.
Examp le 19 .3. ARP t raf f ic f ilt erin g
This example allows chains with names arp-xyz or arp-test to be specified and have their ARP protocol packets evaluated in those chains.
The following filter XML shows an example of filtering ARP traffic in the arp chain.
<filter name='no-arp-spoofing' chain='arp' priority='-500'>
<uuid>f88f1932-debf-4aa1-9fbe-f10d3aa4bc95</uuid>
<rule action='drop' direction='out' priority='300'>
<mac match='no' srcmacaddr='$MAC'/>
</rule>
<rule action='drop' direction='out' priority='350'>
<arp match='no' arpsrcmacaddr='$MAC'/>
</rule>
<rule action='drop' direction='out' priority='400'>
<arp match='no' arpsrcipaddr='$IP'/>
</rule>
<rule action='drop' direction='in' priority='450'>
<arp opcode='Reply'/>
<arp match='no' arpdstmacaddr='$MAC'/>
</rule>
<rule action='drop' direction='in' priority='500'>
<arp match='no' arpdstipaddr='$IP'/>
</rule>
<rule action='accept' direction='inout' priority='600'>
<arp opcode='Request'/>
289
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
</rule>
<rule action='accept' direction='inout' priority='650'>
<arp opcode='Reply'/>
</rule>
<rule action='drop' direction='inout' priority='1000'/>
</filter>
The consequence of putting ARP-specific rules in the arp chain, rather than for example in the root chain, is that packets protocols other than ARP do not need to be evaluated by ARP protocolspecific rules. This improves the efficiency of the traffic filtering. However, one must then pay attention to only putting filtering rules for the given protocol into the chain since other rules will not be evaluated. For example, an IPv4 rule will not be evaluated in the ARP chain since IPv4 protocol packets will not traverse the ARP chain.
19.10.3. Filt ering chain priorit ies
As previously mentioned, when creating a filtering rule, all chains are connected to the root chain.
The order in which those chains are accessed is influenced by the priority of the chain. The following table shows the chains that can be assigned a priority and their default priorities.
T ab le 19 .1. Filt erin g ch ain d ef au lt p rio rit ies valu es
Ch ain ( p ref ix)
stp mac vlan ipv4 ipv6 arp rarp
Def au lt p rio rit y
-810
-800
-750
-700
-600
-500
-400
Note
A chain with a lower priority value is accessed before one with a higher value.
The chains listed in
Table 19.1, “ Filtering chain default priorities values” can be also be
assigned custom priorities by writing a value in the range [-1000 to 1000] into the priority
(XML) attribute in the filter node. Section 19.10.2, “ Filtering chains”
filter shows the default priority of -500 for arp chains, for example.
19.10.4 . Usage of variables in filt ers
There are two variables that have been reserved for usage by the network traffic filtering subsystem:
MAC and IP.
MAC is designated for the MAC address of the network interface. A filtering rule that references this variable will automatically be replaced with the MAC address of the interface. This works without the user having to explicitly provide the MAC parameter. Even though it is possible to specify the MAC parameter similar to the IP parameter above, it is discouraged since libvirt knows what MAC address an interface will be using.
290
Chapt er 1 9 . Virt ual Net working
The parameter IP represents the IP address that the operating system inside the virtual machine is expected to use on the given interface. The IP parameter is special in so far as the libvirt daemon will try to determine the IP address (and thus the IP parameter's value) that is being used on an interface if the parameter is not explicitly provided but referenced. For current limitations on IP address
and what to expect when using it. The XML file shown in
Section 19.10.2, “ Filtering chains” contains
the filter no-arp-spoofing, which is an example of using a network filter XML to reference the MAC and IP variables.
Note that referenced variables are always prefixed with the character $. The format of the value of a variable must be of the type expected by the filter attribute identified in the XML. In the above example, the IP parameter must hold a legal IP address in standard format. Failure to provide the correct structure will result in the filter variable not being replaced with a value and will prevent a virtual machine from starting or will prevent an interface from attaching when hotplugging is being used.
Some of the types that are expected for each XML attribute are shown in the example Example 19.4,
Examp le 19 .4 . Samp le variab le t yp es
As variables can contain lists of elements, (the variable IP can contain multiple IP addresses that are valid on a particular interface, for example), the notation for providing multiple elements for the
IP variable is:
<devices>
<interface type='bridge'>
<mac address='00:16:3e:5d:c7:9e'/>
<filterref filter='clean-traffic'>
<parameter name='IP' value='10.0.0.1'/>
<parameter name='IP' value='10.0.0.2'/>
<parameter name='IP' value='10.0.0.3'/>
</filterref>
</interface>
</devices>
This XML file creates filters to enable multiple IP addresses per interface. Each of the IP addresses will result in a seperate filtering rule. Therefore using the XML above and the the following rule, three individual filtering rules (one for each IP address) will be created:
<rule action='accept' direction='in' priority='500'>
<tcp srpipaddr='$IP'/>
</rule>
As it is possible to access individual elements of a variable holding a list of elements, a filtering rule like the following accesses the 2nd element of the variable DSTPORTS.
<rule action='accept' direction='in' priority='500'>
<udp dstportstart='$DSTPORTS[1]'/>
</rule>
Examp le 19 .5. Usin g a variet y o f variab les
As it is possible to create filtering rules that represent all possible combinations of rules from different lists using the notation $VAR IABLE[@ <i terato r i d = "x">]. The following rule allows
291
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
a virtual machine to receive traffic on a set of ports, which are specified in DSTPORTS, from the set of source IP address specified in SRCIPADDRESSES. The rule generates all combinations of elements of the variable DSTPORTS with those of SRCIPADDRESSES by using two independent iterators to access their elements.
<rule action='accept' direction='in' priority='500'>
<ip srcipaddr='$SRCIPADDRESSES[@ 1]' dstportstart='$DSTPORTS[@ 2]'/>
</rule>
Assign concrete values to SRCIPADDRESSES and DSTPORTS as shown:
SRCIPADDRESSES = [ 10.0.0.1, 11.1.2.3 ]
DSTPORTS = [ 80, 8080 ]
Assigning values to the variables using $SR C IP AD D R ESSES[@ 1] and $D ST P O R T S[@ 2] would then result in all combinations of addresses and ports being created as shown:
10.0.0.1, 80
10.0.0.1, 8080
11.1.2.3, 80
11.1.2.3, 8080
Accessing the same variables using a single iterator, for example by using the notation
$SR C IP AD D R ESSES[@ 1] and $D ST P O R T S[@ 1], would result in parallel access to both lists and result in the following combination:
10.0.0.1, 80
11.1.2.3, 8080
Note
$VAR IABLE is short-hand for $VAR IABLE[@ 0 ]. The former notation always assumes the role of iterator with i terato r i d = "0 " added as shown in the opening paragraph at the top of this section.
19.10.5. Aut omat ic IP address det ect ion and DHCP snooping
1 9 .1 0 .5 .1 . Int ro duct io n
The detection of IP addresses used on a virtual machine's interface is automatically activated if the variable IP is referenced but no value has been assigned to it. The variable CTRL_IP_LEARNING can be used to specify the IP address learning method to use. Valid values include: any, dhcp, or
none.
The value any instructs libvirt to use any packet to determine the address in use by a virtual machine, which is the default setting if the variable TRL_IP_LEARNING is not set. This method will only detect a single IP address per interface. Once a guest virtual machine's IP address has been detected, its IP network traffic will be locked to that address, if for example, IP address spoofing is prevented by one of its filters. In that case, the user of the VM will not be able to change the IP address on the interface
292
Chapt er 1 9 . Virt ual Net working
inside the guest virtual machine, which would be considered IP address spoofing. When a guest virtual machine is migrated to another host physical machine or resumed after a suspend operation, the first packet sent by the guest virtual machine will again determine the IP address that the guest virtual machine can use on a particular interface.
The value of dhcp instructs libvirt to only honor DHCP server-assigned addresses with valid leases.
This method supports the detection and usage of multiple IP address per interface. When a guest virtual machine resumes after a suspend operation, any valid IP address leases are applied to its filters. Otherwise the guest virtual machine is expected to use DHCP to obtain a new IP addresses.
When a guest virtual machine migrates to another physical host physical machine, the guest virtual machine is required to re-run the DHCP protocol.
If CTRL_IP_LEARNING is set to none, libvirt does not do IP address learning and referencing IP without assigning it an explicit value is an error.
1 9 .1 0 .5 .2 . DHCP sno o ping
C T R L_IP _LEAR NING = dhcp (DHCP snooping) provides additional anti-spoofing security, especially when combined with a filter allowing only trusted DHCP servers to assign IP addresses.
To enable this, set the variable DHCPSERVER to the IP address of a valid DHCP server and provide filters that use this variable to filter incoming DHCP responses.
When DHCP snooping is enabled and the DHCP lease expires, the guest virtual machine will no longer be able to use the IP address until it acquires a new, valid lease from a DHCP server. If the guest virtual machine is migrated, it must get a new valid DHCP lease to use an IP address (e.g., by bringing the VM interface down and up again).
Note
Automatic DHCP detection listens to the DHCP traffic the guest virtual machine exchanges with the DHCP server of the infrastructure. To avoid denial-of-service attacks on libvirt, the evaluation of those packets is rate-limited, meaning that a guest virtual machine sending an excessive number of DHCP packets per second on an interface will not have all of those packets evaluated and thus filters may not get adapted. Normal DHCP client behavior is assumed to send a low number of DHCP packets per second. Further, it is important to setup appropriate filters on all guest virtual machines in the infrastructure to avoid them being able to send DHCP packets. Therefore guest virtual machines must either be prevented from sending UDP and TCP traffic from port 67 to port 68 or the DHCPSERVER variable should be used on all guest virtual machines to restrict DHCP server messages to only be allowed to originate from trusted DHCP servers. At the same time anti-spoofing prevention must be enabled on all guest virtual machines in the subnet.
Examp le 19 .6 . Act ivat in g IPs f o r DHCP sn o o p in g
The following XML provides an example for the activation of IP address learning using the DHCP snooping method:
<interface type='bridge'>
<source bridge='virbr0'/>
<filterref filter='clean-traffic'>
<parameter name='CTRL_IP_LEARNING' value='dhcp'/>
</filterref>
</interface>
293
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
19.10.6. Reserved Variables
Table 19.2, “ Reserved variables” shows the variables that are considered reserved and are used by
libvirt:
T ab le 19 .2. Reserved variab les
Variab le Name
MAC
IP
IPV6
DHCPSERVER
DHCPSERVERV6
CTRL_IP_LEARNING
Def in it io n
The MAC address of the interface
The list of IP addresses in use by an interface
Not currently implemented: the list of IPV6 addresses in use by an interface
The list of IP addresses of trusted DHCP servers
Not currently implemented: The list of IPv6 addresses of trusted DHCP servers
The choice of the IP address detection mode
19.10.7. Element and at t ribut e overview
The root element required for all network filters is named <fi l ter> with two possible attributes. The
name attribute provides a unique name of the given filter. The chai n attribute is optional but allows certain filters to be better organized for more efficient processing by the firewall subsystem of the underlying host physical machine. Currently the system only supports the following chains: ro o t,
i pv4 , i pv6 , arp and rarp.
19.10.8. References t o ot her filt ers
Any filter may hold references to other filters. Individual filters may be referenced multiple times in a filter tree but references between filters must not introduce loops.
Examp le 19 .7. An Examp le o f a clean t raf f ic f ilt er
The following shows the XML of the clean-traffic network filter referencing several other filters.
<filter name='clean-traffic'>
<uuid>6ef53069-ba34-94a0-d33d-17751b9b8cb1</uuid>
<filterref filter='no-mac-spoofing'/>
<filterref filter='no-ip-spoofing'/>
<filterref filter='allow-incoming-ipv4'/>
<filterref filter='no-arp-spoofing'/>
<filterref filter='no-other-l2-traffic'/>
<filterref filter='qemu-announce-self'/>
</filter>
To reference another filter, the XML node filterref needs to be provided inside a filter node. This node must have the attribute filter whose value contains the name of the filter to be referenced.
294
Chapt er 1 9 . Virt ual Net working
New network filters can be defined at any time and may contain references to network filters that are not known to libvirt, yet. However, once a virtual machine is started or a network interface referencing a filter is to be hotplugged, all network filters in the filter tree must be available. Otherwise the virtual machine will not start or the network interface cannot be attached.
19.10.9. Filt er rules
The following XML shows a simple example of a network traffic filter implementing a rule to drop traffic if the IP address (provided through the value of the variable IP) in an outgoing IP packet is not the expected one, thus preventing IP address spoofing by the VM.
Examp le 19 .8. Examp le o f n et wo rk t raf f ic f ilt erin g
<filter name='no-ip-spoofing' chain='ipv4'>
<uuid>fce8ae33-e69e-83bf-262e-30786c1f8072</uuid>
<rule action='drop' direction='out' priority='500'>
<ip match='no' srcipaddr='$IP'/>
</rule>
</filter>
The traffic filtering rule starts with the rule node. This node may contain up to three of the following attributes: action is mandatory can have the following values: drop (matching the rule silently discards the packet with no further analysis) reject (matching the rule generates an ICMP reject message with no further analysis) accept (matching the rule accepts the packet with no further analysis) return (matching the rule passes this filter, but returns control to the calling filter for further analysis) continue (matching the rule goes on to the next rule for further analysis) direction is mandatory can have the following values: in for incomming traffic out for outgoing traffic inout for incoming and outgoing traffic priority is optional. The priority of the rule controls the order in which the rule will be instantiated relative to other rules. Rules with lower values will be instantiated before rules with higher values.
Valid values are in the range of -1000 to 1000. If this attribute is not provided, priority 500 will be assigned by default. Note that filtering rules in the root chain are sorted with filters connected to the root chain following their priorities. This allows to interleave filtering rules with access to filter chains. Refer to
Section 19.10.3, “ Filtering chain priorities” for more information.
statematch is optional. Possible values are '0' or 'false' to turn the underlying connection state matching off. The default setting is 'true' or 1
For more information see
Section 19.10.11, “ Advanced Filter Configuration Topics” .
295
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
ip will be associated with the chain ipv4 and the rule will have pri o ri ty= 500. If for example another filter is referenced whose traffic of type ip is also associated with the chain ipv4 then that filter's rules will be ordered relative to the pri o ri ty= 500 of the shown rule.
A rule may contain a single rule for filtering of traffic. The above example shows that traffic of type ip is to be filtered.
19.10.10. Support ed prot ocols
The following sections list and give some details about the protocols that are supported by the network filtering subsystem. This type of traffic rule is provided in the rule node as a nested node.
Depending on the traffic type a rule is filtering, the attributes are different. The above example showed the single attribute srcipaddr that is valid inside the ip traffic filtering node. The following sections show what attributes are valid and what type of data they are expecting. The following datatypes are available:
UINT8 : 8 bit integer; range 0-255
UINT16: 16 bit integer; range 0-65535
MAC_ADDR: MAC address in dotted decimal format, i.e., 00:11:22:33:44:55
MAC_MASK: MAC address mask in MAC address format, i.e., FF:FF:FF:FC:00:00
IP_ADDR: IP address in dotted decimal format, i.e., 10.1.2.3
IP_MASK: IP address mask in either dotted decimal format (255.255.248.0) or CIDR mask (0-32)
IPV6_ADDR: IPv6 address in numbers format, i.e., FFFF::1
IPV6_MASK: IPv6 mask in numbers format (FFFF:FFFF:FC00::) or CIDR mask (0-128)
STRING: A string
BOOLEAN: 'true', 'yes', '1' or 'false', 'no', '0'
IPSETFLAGS: The source and destination flags of the ipset described by up to 6 'src' or 'dst' elements selecting features from either the source or destination part of the packet header; example: src,src,dst. The number of 'selectors' to provide here depends on the type of ipset that is referenced
Every attribute except for those of type IP_MASK or IPV6_MASK can be negated using the match attribute with value no. Multiple negated attributes may be grouped together. The following XML fragment shows such an example using abstract attributes.
[...]
<rule action='drop' direction='in'>
<protocol match='no' attribute1='value1' attribute2='value2'/>
<protocol attribute3='value3'/>
</rule>
[...]
Rules behave evaluate the rule as well as look at it logically within the boundaries of the given protocol attributes. Thus, if a single attribute's value does not match the one given in the rule, the whole rule will be skipped during the evaluation process. Therefore, in the above example incoming traffic will only be dropped if: the protocol property attribute1 does not match both value1 and the protocol property attribute2 does not match value2 and the protocol property attribute3 matches value3.
296
Chapt er 1 9 . Virt ual Net working
1 9 .1 0 .1 0 .1 . MAC (Et he rne t )
Protocol ID: mac
Rules of this type should go into the root chain.
T ab le 19 .3. MAC p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask dstmacaddr dstmacmask protocolid comment
The filter can be written as such:
Dat at yp e Def in it io n
MAC_ADDR
MAC_MASK
MAC address of sender
Mask applied to MAC address of sender
MAC_ADDR
MAC_MASK
MAC address of destination
Mask applied to MAC address of destination
UINT16 (0x600-0xffff), STRING Layer 3 protocol ID. Valid strings include [arp, rarp, ipv4, ipv6]
STRING text string up to 256 characters
[...]
<mac match='no' srcmacaddr='$MAC'/>
[...]
1 9 .1 0 .1 0 .2 . VLAN (8 0 2 .1 Q)
Protocol ID: vlan
Rules of this type should go either into the root or vlan chain.
T ab le 19 .4 . VLAN p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK dstmacaddr dstmacmask vlan-id encap-protocol
MAC_ADDR
MAC_MASK
UINT16 (0x0-0xfff, 0 - 4095)
UINT16 (0x03c-0xfff), String
Def in it io n
MAC address of sender
Mask applied to MAC address of sender
MAC address of destination
Mask applied to MAC address of destination
VLAN ID
Encapsulated layer 3 protocol
ID, valid strings are arp, ipv4, ipv6 text string up to 256 characters comment STRING
1 9 .1 0 .1 0 .3. ST P (Spanning T re e Pro t o co l)
Protocol ID: stp
Rules of this type should go either into the root or stp chain.
297
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
T ab le 19 .5. ST P p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK type UINT8 flags root-priority root-priority-hi root-address root-address-mask roor-cost root-cost-hi sender-priority-hi sender-address sender-address-mask port port_hi msg-age msg-age-hi max-age-hi hello-time hello-time-hi forward-delay forward-delay-hi comment
UINT16
UINT16
UINT16
UINT16
UINT16
UINT16
UINT16
UINT16
UINT16
STRING
UINT8
UINT16
UINT16 (0x0-0xfff, 0 - 4095)
MAC _ADDRESS
MAC _MASK
UINT32
UINT32
UINT16
MAC_ADDRESS
MAC_MASK
Def in it io n
MAC address of sender
Mask applied to MAC address of sender
Bridge Protocol Data Unit
(BPDU) type
BPDU flagdstmacmask
Root priority range start
Root priority range end root MAC Address root MAC Address mask
Root path cost (range start)
Root path cost range end
Sender prioriry range end
BPDU sender MAC address
BPDU sender MAC address mask
Port identifier (range start)
Port identifier range end
Message age timer (range start)
Message age timer range end
Maximum age time range end
Hello time timer (range start)
Hello time timer range end
Forward delay (range start)
Forward delay range end text string up to 256 characters
1 9 .1 0 .1 0 .4 . ARP/RARP
Protocol ID: arp or rarp
Rules of this type should either go into the root or arp/rarp chain.
T ab le 19 .6 . ARP an d RARP p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK dstmacaddr dstmacmask hwtype protocoltype
MAC_ADDR
MAC_MASK
UINT16
UINT16
Def in it io n
MAC address of sender
Mask applied to MAC address of sender
MAC address of destination
Mask applied to MAC address of destination
Hardware type
Protocol type
298
At t rib u t e Name
opcode arpsrcmacaddr arpdstmacaddr arpsrcipaddr arpdstipaddr gratututous
MAC_ADDR
MAC _ADDR
IP_ADDR
IP_ADDR
BOOLEAN comment STRING
1 9 .1 0 .1 0 .5 . IPv4
Protocol ID: ip
Rules of this type should either go into the root or ipv4 chain.
T ab le 19 .7. IPv4 p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK dstmacaddr dstmacmask
MAC_ADDR
MAC_MASK srcipaddr srcipmask dstipaddr dstipmask protocol
IP_ADDR
IP_MASK
IP_ADDR
IP_MASK
UINT8, STRING srcportstart srcportend dstportstart
Dat at yp e
UINT16, STRING
UINT16
UINT16
UNIT16
Chapt er 1 9 . Virt ual Net working
Def in it io n
Opcode valid strings are:
Request, Reply,
Request_Reverse,
Reply_Reverse,
DRARP_Request,
DRARP_Reply, DRARP_Error,
InARP_Request, ARP_NAK
Source MAC address in
ARP/RARP packet
Destination MAC address in
ARP/RARP packet
Source IP address in
ARP/RARP packet
Destination IP address in
ARP/RARP packet
Boolean indiating whether to check for a gratuitous ARP packet text string up to 256 characters
Def in it io n
MAC address of sender
Mask applied to MAC address of sender
MAC address of destination
Mask applied to MAC address of destination
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address
Layer 4 protocol identifier. Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmp, igmp, sctp
Start of range of valid source ports; requires protocol
End of range of valid source ports; requires protocol
Start of range of valid destination ports; requires protocol
299
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
At t rib u t e Name Dat at yp e Def in it io n
dstportend UNIT16 End of range of valid destination ports; requires protocol text string up to 256 characters comment STRING
1 9 .1 0 .1 0 .6 . IPv6
Protocol ID: ipv6
Rules of this type should either go into the root or ipv6 chain.
T ab le 19 .8. IPv6 p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK dstmacaddr dstmacmask srcipaddr srcipmask dstipaddr dstipmask
MAC_ADDR
MAC_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_MASK protocol UINT8, STRING scrportstart srcportend dstportstart dstportend comment
UNIT16
UINT16
UNIT16
UNIT16
STRING
Def in it io n
MAC address of sender
Mask applied to MAC address of sender
MAC address of destination
Mask applied to MAC address of destination
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address
Layer 4 protocol identifier. Valid strings for protocol are: tcp, udp, udplite, esp, ah, icmpv6, sctp
Start of range of valid source ports; requires protocol
End of range of valid source ports; requires protocol
Start of range of valid destination ports; requires protocol
End of range of valid destination ports; requires protocol text string up to 256 characters
1 9 .1 0 .1 0 .7 . T CP/UDP/SCT P
Protocol ID: tcp, udp, sctp
The chain parameter is ignored for this type of traffic and should either be omitted or set to root. .
T ab le 19 .9 . T CP/UDP/SCT P p ro t o co l t yp es
300
Chapt er 1 9 . Virt ual Net working
At t rib u t e Name
srcmacaddr srcipaddr srcipmask dstipaddr dstipmask scripto srcipfrom dstipfrom dstipto scrportstart srcportend dstportstart dstportend comment state flags ipset ipsetflags
Dat at yp e
MAC_ADDR
IP_ADDR
IP_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_ADDR
IP_ADDR
IP_ADDR
UNIT16
UINT16
UNIT16
UNIT16
STRING
STRING
STRING
STRING
IPSETFLAGS
Def in it io n
MAC address of sender
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address
Start of range of source IP address
End of range of source IP address
Start of range of destination IP address
End of range of destination IP address
Start of range of valid source ports; requires protocol
End of range of valid source ports; requires protocol
Start of range of valid destination ports; requires protocol
End of range of valid destination ports; requires protocol text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
TCP-only: format of mask/flags with mask and flags each being a comma separated list of
SYN,ACK,URG,PSH,FIN,RST or
NONE or ALL
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
1 9 .1 0 .1 0 .8 . ICMP
Protocol ID: icmp
Note: The chain parameter is ignored for this type of traffic and should either be omitted or set to root.
T ab le 19 .10. ICMP p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK
Def in it io n
MAC address of sender
Mask applied to the MAC address of the sender
301
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
At t rib u t e Name
dstmacaddr dstmacmask srcipaddr srcipmask dstipaddr dstipmask srcipfrom scripto dstipfrom dstipto type code comment state ipset ipsetflags
Dat at yp e
MAD_ADDR
MAC_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_ADDR
IP_ADDR
IP_ADDR
UNIT16
UNIT16
STRING
STRING
STRING
IPSETFLAGS
Def in it io n
MAC address of the destination
Mask applied to the MAC address of the destination
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address start of range of source IP address end of range of source IP address
Start of range of destination IP address
End of range of destination IP address
ICMP type
ICMP code text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
1 9 .1 0 .1 0 .9 . IGMP, ESP, AH, UDPLIT E, 'ALL'
Protocol ID: igmp, esp, ah, udplite, all
The chain parameter is ignored for this type of traffic and should either be omitted or set to root.
T ab le 19 .11. IG MP, ESP, AH, UDPLIT E, ' ALL'
At t rib u t e Name
srcmacaddr srcmacmask
Dat at yp e
MAC_ADDR
MAC_MASK dstmacaddr dstmacmask srcipaddr srcipmask dstipaddr dstipmask srcipfrom
MAD_ADDR
MAC_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_MASK
IP_ADDR
Def in it io n
MAC address of sender
Mask applied to the MAC address of the sender
MAC address of the destination
Mask applied to the MAC address of the destination
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address start of range of source IP address
302
Chapt er 1 9 . Virt ual Net working
At t rib u t e Name
scripto dstipfrom dstipto comment state ipset ipsetflags
Dat at yp e
IP_ADDR
IP_ADDR
IP_ADDR
STRING
STRING
STRING
IPSETFLAGS
Def in it io n
end of range of source IP address
Start of range of destination IP address
End of range of destination IP address text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
1 9 .1 0 .1 0 .1 0 . T CP/UDP/SCT P o ve r IPV6
Protocol ID: tcp-ipv6, udp-ipv6, sctp-ipv6
The chain parameter is ignored for this type of traffic and should either be omitted or set to root.
T ab le 19 .12. T CP, UDP, SCT P o ver IPv6 p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcipaddr srcipmask
Dat at yp e
MAC_ADDR
IP_ADDR
IP_MASK dstipaddr dstipmask srcipfrom scripto dstipfrom dstipto srcportstart srcportend dstportstart dstportend comment state
IP_ADDR
IP_MASK
IP_ADDR
IP_ADDR
IP_ADDR
IP_ADDR
UINT16
UINT16
UINT16
UINT16
STRING
STRING
Def in it io n
MAC address of sender
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address start of range of source IP address end of range of source IP address
Start of range of destination IP address
End of range of destination IP address
Start of range of valid source ports
End of range of valid source ports
Start of range of valid destination ports
End of range of valid destination ports text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
303
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
At t rib u t e Name
ipset ipsetflags
Dat at yp e
STRING
IPSETFLAGS
Def in it io n
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
1 9 .1 0 .1 0 .1 1 . ICMPv6
Protocol ID: icmpv6
The chain parameter is ignored for this type of traffic and should either be omitted or set to root.
T ab le 19 .13. ICMPv6 p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcipaddr srcipmask
Dat at yp e
MAC_ADDR
IP_ADDR
IP_MASK dstipaddr dstipmask srcipfrom scripto dstipfrom dstipto type code comment state
IP_ADDR
IP_MASK
IP_ADDR
IP_ADDR
IP_ADDR
IP_ADDR
UINT16
UINT16
STRING
STRING ipset ipsetflags
STRING
IPSETFLAGS
Def in it io n
MAC address of sender
Source IP address
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address start of range of source IP address end of range of source IP address
Start of range of destination IP address
End of range of destination IP address
ICMPv6 type
ICMPv6 code text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
1 9 .1 0 .1 0 .1 2 . IGMP, ESP, AH, UDPLIT E, 'ALL' o ve r IPv6
Protocol ID: igmp-ipv6, esp-ipv6, ah-ipv6, udplite-ipv6, all-ipv6
The chain parameter is ignored for this type of traffic and should either be omitted or set to root.
T ab le 19 .14 . IG MP, ESP, AH, UDPLIT E, ' ALL' o ver IPv p ro t o co l t yp es
At t rib u t e Name
srcmacaddr srcipaddr
Dat at yp e
MAC_ADDR
IP_ADDR
Def in it io n
MAC address of sender
Source IP address
304
Chapt er 1 9 . Virt ual Net working
At t rib u t e Name
srcipmask dstipaddr dstipmask srcipfrom scripto dstipfrom dstipto comment state ipset ipsetflags
Dat at yp e
IP_MASK
IP_ADDR
IP_MASK
IP_ADDR
IP_ADDR
IP_ADDR
IP_ADDR
STRING
STRING
STRING
IPSETFLAGS
Def in it io n
Mask applied to source IP address
Destination IP address
Mask applied to destination IP address start of range of source IP address end of range of source IP address
Start of range of destination IP address
End of range of destination IP address text string up to 256 characters comma separated list of
NEW,ESTABLISHED,RELATED,I
NVALID or NONE
The name of an IPSet managed outside of libvirt flags for the IPSet; requires ipset attribute
19.10.11. Advanced Filt er Configurat ion T opics
The following sections discuss advanced filter configuration topics.
1 9 .1 0 .1 1 .1 . Co nne ct io n t racking
The network filtering subsystem (on Linux) makes use of the connection tracking support of IP tables.
This helps in enforcing the directionality of network traffic (state match) as well as counting and limiting the number of simultaneous connections towards a guest virtual machine. As an example, if a guest virtual machine has TCP port 8080 open as a server, clients may connect to the guest virtual machine on port 8080. Connection tracking and enforcement of directionality then prevents the guest virtual machine from initiating a connection from (TCP client) port 8080 to the host physical machine back to a remote host physical machine. More importantly, tracking helps to prevent remote attackers from establishing a connection back to a guest virtual machine. For example, if the user inside the guest virtual machine established a connection to port 80 on an attacker site, then the attacker will not be able to initiate a connection from TCP port 80 back towards the guest virtual machine. By default the connection state match that enables connection tracking and then enforcement of directionality of traffic is turned on.
Examp le 19 .9 . XML examp le f o r t u rn in g o f f co n n ect io n s t o t h e T CP p o rt
The following shows an example XML fragment where this feature has been turned off for incoming connections to TCP port 12345.
[...]
<rule direction='in' action='accept' statematch='false'>
<cp dstportstart='12345'/>
</rule>
[...]
305
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
This now allows incoming traffic to TCP port 12345, but would also enable the initiation from
(client) TCP port 12345 within the VM, which may or may not be desirable.
1 9 .1 0 .1 1 .2 . Lim it ing Num be r o f Co nne ct io ns
To limit the number of connections a guest virtual machine may establish, a rule must be provided that sets a limit of connections for a given type of traffic. If for example a VM is supposed to be allowed to only ping one other IP address at a time and is supposed to have only one active incoming ssh connection at a time.
Examp le 19 .10. XML samp le f ile t h at set s limit s t o co n n ect io n s
The following XML fragment can be used to limit connections
[...]
<rule action='drop' direction='in' priority='400'>
<tcp connlimit-above='1'/>
</rule>
<rule action='accept' direction='in' priority='500'>
<tcp dstportstart='22'/>
</rule>
<rule action='drop' direction='out' priority='400'>
<icmp connlimit-above='1'/>
</rule>
<rule action='accept' direction='out' priority='500'>
<icmp/>
</rule>
<rule action='accept' direction='out' priority='500'>
<udp dstportstart='53'/>
</rule>
<rule action='drop' direction='inout' priority='1000'>
<all/>
</rule>
[...]
306
Chapt er 1 9 . Virt ual Net working
Note
Limitation rules must be listed in the XML prior to the rules for accepting traffic. According to the XML file in
Example 19.10, “ XML sample file that sets limits to connections”
, an additional rule for allowing DNS traffic sent to port 22 go out the guest virtual machine, has been added to avoid ssh sessions not getting established for reasons related to DNS lookup failures by the ssh daemon. Leaving this rule out may result in the ssh client hanging unexpectedly as it tries to connect. Additional caution should be used in regards to handling timeouts related to tracking of traffic. An ICMP ping that the user may have terminated inside the guest virtual machine may have a long timeout in the host physical machine's connection tracking system and will therefore not allow another ICMP ping to go through.
The best solution is to tune the timeout in the host physical machine's sysfs with the following command:# echo 3 >
/pro c/sys/net/netfi l ter/nf_co nntrack_i cmp_ti meo ut. This command sets the
ICMP connection tracking timeout to 3 seconds. The effect of this is that once one ping is terminated, another one can start after 3 seconds.
If for any reason the guest virtual machine has not properly closed its TCP connection, the connection to be held open for a longer period of time, especially if the TCP timeout value was set for a large amount of time on the host physical machine. In addition, any idle connection may result in a time out in the connection tracking system which can be re-activated once packets are exchanged.
However, if the limit is set too low, newly initiated connections may force an idle connection into TCP backoff. Therefore, the limit of connections should be set rather high so that fluctuations in new TCP connections don't cause odd traffic behavior in relation to idle connections.
1 9 .1 0 .1 1 .3. Co m m and line t o o ls
virsh has been extended with life-cycle support for network filters. All commands related to the network filtering subsystem start with the prefix nwfilter. The following commands are available:
nwfi l ter-l i st : lists UUIDs and names of all network filters
nwfi l ter-d efi ne : defines a new network filter or updates an existing one (must supply a name)
nwfi l ter-und efi ne : deletes a specified network filter (must supply a name). Do not delete a network filter currently in use.
nwfi l ter-d umpxml : displays a specified network filter (must supply a name)
nwfi l ter-ed i t : edits a specified network filter (must supply a name)
1 9 .1 0 .1 1 .4 . Pre -e xist ing ne t wo rk filt e rs
The following is a list of example network filters that are automatically installed with libvirt:
T ab le 19 .15. ICMPv6 p ro t o co l t yp es
Co mman d Name Descrip t io n
307
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Co mman d Name
no-arp-spoofing allow-dhcp allow-dhcp-server no-ip-spoofing no-ip-multicast clean-traffic
Descrip t io n
Prevents a guest virtual machine from spoofing
ARP traffic; this filter only allows ARP request and reply messages and enforces that those packets contain the MAC and IP addresses of the guest virtual machine.
Allows a guest virtual machine to request an IP address via DHCP (from any DHCP server)
Allows a guest virtual machine to request an IP address from a specified DHCP server. The dotted decimal IP address of the DHCP server must be provided in a reference to this filter. The name of the variable must be DHCPSERVER.
Prevents a guest virtual machine from sending
IP packets with a source IP address different from the one inside the packet.
Prevents a guest virtual machine from sending
IP multicast packets.
Prevents MAC, IP and ARP spoofing. This filter references several other filters as building blocks.
These filters are only building blocks and require a combination with other filters to provide useful network traffic filtering. The most used one in the above list is the clean-traffic filter. This filter itself can for example be combined with the no-ip-multicast filter to prevent virtual machines from sending IP multicast traffic on top of the prevention of packet spoofing.
1 9 .1 0 .1 1 .5 . Writ ing yo ur o wn filt e rs
Since libvirt only provides a couple of example networking filters, you may consider writing your own.
When planning on doing so there are a couple of things you may need to know regarding the network filtering subsystem and how it works internally. Certainly you also have to know and understand the protocols very well that you want to be filtering on so that no further traffic than what you want can pass and that in fact the traffic you want to allow does pass.
The network filtering subsystem is currently only available on Linux host physical machines and only works for Qemu and KVM type of virtual machines. On Linux, it builds upon the support for ebtables, iptables and ip6tables and makes use of their features. Considering the list found in
Section 19.10.10, “ Supported protocols” the following protocols can be implemented using ebtables:
mac stp (spanning tree protocol) vlan (802.1Q) arp, rarp ipv4 ipv6
Any protocol that runs over IPv4 is supported using iptables, those over IPv6 are implemented using ip6tables.
308
Chapt er 1 9 . Virt ual Net working
Using a Linux host physical machine, all traffic filtering rules created by libvirt's network filtering subsystem first passes through the filtering support implemented by ebtables and only afterwards through iptables or ip6tables filters. If a filter tree has rules with the protocols including: mac, stp, vlan arp, rarp, ipv4, or ipv6; the ebtable rules and values listed will automatically be used first.
Multiple chains for the same protocol can be created. The name of the chain must have a prefix of one of the previously enumerated protocols. To create an additional chain for handling of ARP traffic, a chain with name arp-test, can for example be specified.
As an example, it is possible to filter on UDP traffic by source and destination ports using the IP protocol filter and specifying attributes for the protocol, source and destination IP addresses and ports of UDP packets that are to be accepted. This allows early filtering of UDP traffic with ebtables.
However, once an IP or IPv6 packet, such as a UDP packet, has passed the ebtables layer and there is at least one rule in a filter tree that instantiates iptables or ip6tables rules, a rule to let the UDP packet pass will also be necessary to be provided for those filtering layers. This can be achieved with a rule containing an appropriate udp or udp-ipv6 traffic filtering node.
Examp le 19 .11. Creat in g a cu st o m f ilt er
Suppose a filter is needed to fulfill the following list of requirements: prevents a VM's interface from MAC, IP and ARP spoofing opens only TCP ports 22 and 80 of a VM's interface allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface allows the VM to do DNS lookups (UDP towards port 53)
The requirement to prevent spoofing is fulfilled by the existing clean-traffic network filter, thus the way to do this is to reference it from a custom filter.
To enable traffic for TCP ports 22 and 80, two rules are added to enable this type of traffic. To allow the guest virtual machine to send ping traffic a rule is added for ICMP traffic. For simplicity reasons, general ICMP traffic will be allowed to be initiated from the guest virtual machine, and will not be specified to ICMP echo request and response messages. All other traffic will be prevented to reach or be initiated by the guest virtual machine. To do this a rule will be added that drops all other traffic. Assuming the guest virtual machine is called test and the interface to associate our filter with is called eth0 , a filter is created named test-eth0 .
The result of these considerations is the following network filter XML:
<filter name='test-eth0'>
<!- - This rule references the clean traffic filter to prevent MAC,
IP and ARP spoofing. By not providing an IP address parameter, libvirt will detect the IP address the guest virtual machine is using. - ->
<filterref filter='clean-traffic'/>
<!- - This rule enables TCP ports 22 (ssh) and 80 (http) to be reachable - ->
<rule action='accept' direction='in'>
<tcp dstportstart='22'/>
</rule>
<rule action='accept' direction='in'>
<tcp dstportstart='80'/>
</rule>
309
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine including ping traffic - ->
<rule action='accept' direction='out'>
<icmp/>
</rule>>
<!- - This rule enables outgoing DNS lookups using UDP - ->
<rule action='accept' direction='out'>
<udp dstportstart='53'/>
</rule>
<!- - This rule drops all other traffic - ->
<rule action='drop' direction='inout'>
<all/>
</rule>
</filter>
1 9 .1 0 .1 1 .6 . Sam ple cust o m filt e r
Although one of the rules in the above XML contains the IP address of the guest virtual machine as either a source or a destination address, the filtering of the traffic works correctly. The reason is that whereas the rule's evaluation occurs internally on a per-interface basis, the rules are additionally evaluated based on which (tap) interface has sent or will receive the packet, rather than what their source or destination IP address may be.
Examp le 19 .12. Samp le XML f o r n et wo rk in t erf ace d escrip t io n s
An XML fragment for a possible network interface description inside the domain XML of the test guest virtual machine could then look like this:
[...]
<interface type='bridge'>
<source bridge='mybridge'/>
<filterref filter='test-eth0'/>
</interface>
[...]
To more strictly control the ICMP traffic and enforce that only ICMP echo requests can be sent from the guest virtual machine and only ICMP echo responses be received by the guest virtual machine, the above ICMP rule can be replaced with the following two rules:
<!- - enable outgoing ICMP echo requests- ->
<rule action='accept' direction='out'>
<icmp type='8'/>
</rule>
<!- - enable incoming ICMP echo replies- ->
<rule action='accept' direction='in'>
<icmp type='0'/>
</rule>
310
Chapt er 1 9 . Virt ual Net working
Examp le 19 .13. Seco n d examp le cu st o m f ilt er
This example demonstrates how to build a similar filter as in the example above, but extends the list of requirements with an ftp server located inside the guest virtual machine. The requirements for this filter are: prevents a guest virtual machine's interface from MAC, IP, and ARP spoofing opens only TCP ports 22 and 80 in a guest virtual machine's interface allows the guest virtual machine to send ping traffic from an interface but does not allow the guest virtual machine to be pinged on the interface allows the guest virtual machine to do DNS lookups (UDP towards port 53) enables the ftp server (in active mode) so it can run inside the guest virtual machine
The additional requirement of allowing an FTP server to be run inside the guest virtual machine maps into the requirement of allowing port 21 to be reachable for FTP control traffic as well as enabling the guest virtual machine to establish an outgoing TCP connection originating from the guest virtual machine's TCP port 20 back to the FTP client (FTP active mode). There are several ways of how this filter can be written and two possible solutions are included in this example.
The first solution makes use of the state attribute of the TCP protocol that provides a hook into the connection tracking framework of the Linux host physical machine. For the guest virtual machineinitiated FTP data connection (FTP active mode) the RELATED state is used to enable detection that the guest virtual machine-initiated FTP data connection is a consequence of ( or 'has a relationship with' ) an existing FTP control connection, thereby allowing it to pass packets through the firewall. The RELATED state, however, is only valid for the very first packet of the outgoing TCP connection for the FTP data path. Afterwards, the state is ESTABLISHED, which then applies equally to the incoming and outgoing direction. All this is related to the FTP data traffic originating from TCP port 20 of the guest virtual machine. This then leads to the following solution:
<filter name='test-eth0'>
<!- - This filter (eth0) references the cl ean traffi c filter to prevent MAC, IP, and ARP spoofing. By not providing an IP address parameter, libvirt will detect the IP address the guest virtual machine is using. - ->
<filterref filter='clean-traffic'/>
<!- - This rule enables TCP port 21 (FTP-control) to be reachable -
->
<rule action='accept' direction='in'>
<tcp dstportstart='21'/>
</rule>
<!- - This rule enables TCP port 20 for guest virtual machineinitiated FTP data connection related to an existing FTP control connection - ->
<rule action='accept' direction='out'>
<tcp srcportstart='20' state='RELATED,ESTABLISHED'/>
</rule>
<!- - This rule accepts all packets from a client on the FTP data connection - ->
311
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<rule action='accept' direction='in'>
<tcp dstportstart='20' state='ESTABLISHED'/>
</rule>
<!- - This rule enables TCP port 22 (SSH) to be reachable - ->
<rule action='accept' direction='in'>
<tcp dstportstart='22'/>
</rule>
<!- -This rule enables TCP port 80 (HTTP) to be reachable - ->
<rule action='accept' direction='in'>
<tcp dstportstart='80'/>
</rule>
<!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine, including ping traffic - ->
<rule action='accept' direction='out'>
<icmp/>
</rule>
<!- - This rule enables outgoing DNS lookups using UDP - ->
<rule action='accept' direction='out'>
<udp dstportstart='53'/>
</rule>
<!- - This rule drops all other traffic - ->
<rule action='drop' direction='inout'>
<all/>
</rule>
</filter>
Before trying out a filter using the RELATED state, you have to make sure that the appropriate connection tracking module has been loaded into the host physical machine's kernel. Depending on the version of the kernel, you must run either one of the following two commands before the FTP connection with the guest virtual machine is established:
#mo d pro be nf_co nntrack_ftp - where available OR
#mo d pro be i p_co nntrack_ftp if above is not available
If protocols other than FTP are used in conjunction with the RELATED state, their corresponding module must be loaded. Modules are available for the protocols: ftp, tftp, irc, sip, sctp, and amanda.
The second solution makes use of the state flags of connections more than the previous solution did. This solution takes advantage of the fact that the NEW state of a connection is valid when the very first packet of a traffic flow is detected. Subsequently, if the very first packet of a flow is accepted, the flow becomes a connection and thus enters into the ESTABLISHED state. Therefore a general rule can be written for allowing packets of ESTABLISHED connections to reach the guest virtual machine or be sent by the guest virtual machine. This is done writing specific rules for the very first packets identified by the NEW state and dictates the ports that the data is acceptable. All packets meant for ports that are not explicitly accepted are dropped, thus not reaching an
ESTABLISHED state. Any subsequent packets sent from that port are dropped as well.
<filter name='test-eth0'>
<!- - This filter references the cl ean traffi c filter to prevent MAC,
312
Chapt er 1 9 . Virt ual Net working
IP and ARP spoofing. By not providing and IP address parameter, libvirt will detect the IP address the VM is using. - ->
<filterref filter='clean-traffic'/>
<!- - This rule allows the packets of all previously accepted connections to reach the guest virtual machine - ->
<rule action='accept' direction='in'>
<all state='ESTABLISHED'/>
</rule>
<!- - This rule allows the packets of all previously accepted and related connections be sent from the guest virtual machine - ->
<rule action='accept' direction='out'>
<all state='ESTABLISHED,RELATED'/>
</rule>
<!- - This rule enables traffic towards port 21 (FTP) and port 22
(SSH)- ->
<rule action='accept' direction='in'>
<tcp dstportstart='21' dstportend='22' state='NEW'/>
</rule>
<!- - This rule enables traffic towards port 80 (HTTP) - ->
<rule action='accept' direction='in'>
<tcp dstportstart='80' state='NEW'/>
</rule>
<!- - This rule enables general ICMP traffic to be initiated by the guest virtual machine, including ping traffic - ->
<rule action='accept' direction='out'>
<icmp state='NEW'/>
</rule>
<!- - This rule enables outgoing DNS lookups using UDP - ->
<rule action='accept' direction='out'>
<udp dstportstart='53' state='NEW'/>
</rule>
<!- - This rule drops all other traffic - ->
<rule action='drop' direction='inout'>
<all/>
</rule>
</filter>
19.10.12. Limit at ions
The following is a list of the currently known limitations of the network filtering subsystem.
VM migration is only supported if the whole filter tree that is referenced by a guest virtual machine's top level filter is also available on the target host physical machine. The network filter
cl ean-traffi c for example should be available on all libvirt installations and thus enable migration of guest virtual machines that reference this filter. To assure version compatibility is not a problem make sure you are using the most current version of libvirt by updating the package regularly.
313
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Migration must occur between libvirt installations of version 0.8.1 or later in order not to lose the network traffic filters associated with an interface.
VLAN (802.1Q) packets, if sent by a guest virtual machine, cannot be filtered with rules for protocol IDs arp, rarp, ipv4 and ipv6. They can only be filtered with protocol IDs, MAC and VLAN.
Therefore, the example filter clean-traffic
Example 19.1, “ An example of network filtering” will not
work as expected.
19.11. Creat ing T unnels
This section will demonstrate how to implement different tunneling scenarios.
19.11.1. Creat ing Mult icast t unnels
A multicast group is setup to represent a virtual network. Any guest virtual machines whose network devices are in the same multicast group can talk to each other even across host physical machines.
This mode is also available to unprivileged users. There is no default DNS or DHCP support and no outgoing network access. To provide outgoing network access, one of the guest virtual machines should have a second NIC which is connected to one of the first four network types thus providing appropriate routing. The multicast protocol is compatible the guest virtual machine user mode. Note that the source address that you provide must be from the address used for the multicast address block.
To create a multicast tunnel place the following XML details into the <d evi ces> element:
...
<devices>
<interface type= 'mcast' >
<mac address= '52:54:00:6d:90:01' >
<source address= '230.0.0.1' port= '5558' />
</interface>
</devices>
...
Fig u re 19 .25. Mu lt icast t u n n el d o main XMl examp le
19.11.2. Creat ing T CP t unnels
A TCP client/server architecture provides a virtual network. In this configuration, one guest virtual machine provides the server end of the network while all other guest virtual machines are configured as clients. All network traffic is routed between the guest virtual machine clients via the guest virtual machine server. This mode is also available for unprivileged users. Note that this mode does not provide default DNS or DHCP support nor does it provide outgoing network access. To provide outgoing network access, one of the guest virtual machines should have a second NIC which is connected to one of the first four network types thus providing appropriate routing.
To create a TCP tunnel place the following XML details into the <d evi ces> element:
314
Chapt er 1 9 . Virt ual Net working
...
<devices>
<interface type= 'server' >
<mac address= '52:54:00:22:c9:42' >
<source address= '192.168.0.1' port= '5558' />
</interface>
...
<interface type= 'client' >
<mac address= '52:54:00:8b:c9:51' >
<source address= '192.168.0.1' port= '5558' />
</interface>
</devices>
...
Fig u re 19 .26 . T CP t u n n el d o main XMl examp le
19.12. Set t ing vLAN t ags
virtual local area network (vLAN) tags are added using the vi rsh net-ed i t command. This tag can also be used with PCI device assignment with SR-IOV devices. For more information, refer to
Section 10.1.7, “ Configuring PCI assignment (passthrough) with SR-IOV devices” .
<network>
<name> ovs-net </name>
<forward mode= 'bridge' />
<bridge name= 'ovsbr0' />
<virtualport type= 'openvswitch' >
<parameters interfaceid= '09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f' />
</virtualport>
<vlan trunk= 'yes' >
<tag id= '42' nativeMode= 'untagged' />
<tag id= '47' />
</vlan>
<portgroup name= 'dontpanic' >
<vlan>
<tag id= '42' />
</vlan>
</portgroup>
</network>
Fig u re 19 .27. vSet t in g VLAN t ag ( o n su p p o rt ed n et wo rk t yp es o n ly)
315
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
If (and only if) the network type supports vlan tagging transparent to the guest, an optional <vl an> element can specify one or more vlan tags to apply to the traffic of all guests using this network.
(openvswitch and type='hostdev' SR-IOV networks do support transparent VLAN tagging of guest traffic; everything else, including standard linux bridges and libvirt's own virtual networks, do not support it. 802.1Qbh (vn-link) and 802.1Qbg (VEPA) switches provide their own way (outside of libvirt) to tag guest traffic onto specific vlans.) As expected, the tag attribute specifies which vlan tag to use. If a network has more than one <vl an> element defined, it is assumed that the user wants to do VLAN trunking using all the specified tags. In the case that VLAN trunking with a single tag is desired, the optional attribute trunk='yes' can be added to the VLAN element.
For network connections using openvswitch it is possible to configure the 'native-tagged' and
'native-untagged' VLAN modes. This uses the optional nativeMode attribute on the <tag > element: nativeMode may be set to 'tagged' or 'untagged'. The id attribute of the element sets the native vlan.
<vl an> elements can also be specified in a <po rtg ro up> element, as well as directly in a domain's
<i nterface> element. In the case that a vlan tag is specified in multiple locations, the setting in
<i nterface> takes precedence, followed by the setting in the <po rtg ro up> selected by the interface config. The <vl an> in <netwo rk> will be selected only if none is given in <po rtg ro up> or
<i nterface>.
19.13. Applying QoS t o your virt ual net work
Quality of Service (QoS) refers to the resource control systems that guarantees an optimal experience for all users on a network, making sure that there is no delay, jitter, or packet loss. QoS can be
application specific or user / group specific. Refer to Section 21.16.9.14, “ Quality of service”
for more information.
316
Chapt er 2 0 . qemu- kvm Whit elist
Chapter 20. qemu-kvm Whitelist
20.1. Int roduct ion
The primary objective of this QEMU-KVM whitelist is to provide a complete list of the supported options of the q emu-kvm utility used as an emulator and a virtualizer in Red Hat Enterprise Linux 6.
This is a comprehensive summary of the supported options. Red Hat Enterprise Linux 6 uses KVM as an underlying virtualization technology. The machine emulator and virtualizer used is a modified version of QEMU called qemu-kvm. This version does not support all configuration options of the original QEMU and it also adds some additional options.
Note
This chapter lists only the supported options of the qemu-kvm utility. Options n o t listed here are not supported by Red Hat.
Whit e list fo rm at
<n ame> - When used in a syntax description, this string should be replaced by user-defined value.
[ a|b |c] - When used in a syntax description, only one of the strings separated by | is used.
When no comment is present, an option is supported with all possible values.
20.2. Basic opt ions
Em ulat e d m achine
- M <machine-type>
- mach in e <machine-type>[,<pro perty>[= <val ue>][,. . ]]
Pro ce sso r t ype
- cp u <model>[,<FEATURE>][...]
Additional models are visible by running -cpu ? command.
O p t ero n _G 5 - AMD Opteron 63xx class CPU
O p t ero n _G 4 - AMD Opteron 62xx class CPU
O p t ero n _G 3 - AMD Opteron 23xx (AMD Opteron Gen 3)
O p t ero n _G 2 - AMD Opteron 22xx (AMD Opteron Gen 2)
O p t ero n _G 1 - AMD Opteron 240 (AMD Opteron Gen 1)
West mere - Westmere E56xx/L56xx/X56xx (Nehalem-C)
Haswell - Intel Core Processor (Haswell)
317
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
San d yBrid g e - Intel Xeon E312xx (Sandy Bridge)
Neh alem - Intel Core i7 9xx (Nehalem Class Core i7)
Pen ryn - Intel Core 2 Duo P9xxx (Penryn Class Core 2)
Co n ro e - Intel Celeron_4x0 (Conroe/Merom Class Core 2)
cp u 6 4 - rh el5 - Red Hat Enterprise Linux 5 supported QEMU Virtual CPU version
cp u 6 4 - rh el6 - Red Hat Enterprise Linux 6 supported QEMU Virtual CPU version
d ef au lt - special option use default option from above.
Pro ce sso r T o po lo gy
- smp <n>[,cores=<ncores>][,threads=<nthreads>][,sockets=<nsocks>][,maxcpus=<maxcpus>]
Hypervisor and guest operating system limits on processor topology apply.
NUMA syst e m
- n u ma <nodes>[,mem=<size>][,cpus=<cpu[-cpu>]][,nodeid=<node>]
Hypervisor and guest operating system limits on processor topology apply.
Me m o ry size
- m <megs>
Supported values are limited by guest minimal and maximal values and hypervisor limits.
Ke ybo ard layo ut
- k <language>
Gue st nam e
- n ame <name>
Gue st UUID
- u u id <uuid>
20.3. Disk opt ions
Ge ne ric drive
- d rive <option>[,<option>[,<option>[,...]]]
Supported with the following options:
read o n ly[on|off]
werro r[enospc|report|stop|ignore]
rerro r[report|stop|ignore]
318
Chapt er 2 0 . qemu- kvm Whit elist
id =<id>
Id of the drive has the following limitation for if=none:
IDE disk has to have <id> in following format: drive-ide0-<BUS>-<UNIT>
Example of correct format:
-drive if=none,id=drive-ide0-<BUS>-<UNIT>,... -device ide-drive,drive=drive-ide0-<BUS>-
<UNIT>,bus=ide.<BUS>,unit=<UNIT>
f ile=<file>
Value of <file> is parsed with the following rules:
Passing floppy device as <file> is not supported.
Passing cd-rom device as <file> is supported only with cdrom media type (media=cdrom) and only as IDE drive (either if=ide or if=none + -device ide-drive).
If <file> is neither block nor character device, it must not contain ':'.
if =<interface>
The following interfaces are supported: none, ide, virtio, floppy.
in d ex=<index>
med ia=<media>
cach e=<cache>
Supported values: none, writeback or writethrough.
co p y- o n - read =[on|off]
sn ap sh o t =[yes|no]
serial=<serial>
aio =<aio>
f o rmat =<format>
This option is not required and can be omitted. However, this is not recommended for raw images because it represents security risk. Supported formats are:
q co w2 raw
Bo o t o pt io n
- b o o t [order=<drives>][,menu=[on|off]]
Snapsho t m o de
- sn ap sh o t
20.4 . Display opt ions
319
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Disable graphics
- n o g rap h ic
VGA card e m ulat io n
- vg a <type>
Supported types:
cirru s - Cirrus Logic GD5446 Video card.
st d - Standard VGA card with Bochs VBE extensions.
q xl - Spice paravirtual card.
n o n e - Disable VGA card.
VNC display
- vn c <display>[,<option>[,<option>[,...]]]
Supported display value:
[<host>]:<port> unix:<path>
sh are[allow-exclusive|force-shared|ignore] none - Supported with no other options specified.
Supported options are:
t o =<port>
reverse p asswo rd t ls
x509 =</path/to/certificate/dir> - Supported when t ls specified.
x509 verif y=</path/to/certificate/dir> - Supported when t ls specified.
sasl acl
Spice de skt o p
- sp ice option[,option[,...]]
Supported options are:
p o rt =<number>
ad d r=<addr>
ip v4
320
ip v6
p asswo rd =<secret>
d isab le- t icket in g d isab le- co p y- p ast e
t ls- p o rt =<number>
x509 - d ir=</path/to/certificate/dir>
x509 - key- f ile=<file>
x509 - key- p asswo rd =<file>
x509 - cert - f ile=<file>
x509 - cacert - f ile=<file>
x509 - d h - key- f ile=<file>
t ls- cip h er=<list>
t ls- ch an n el[main|display|cursor|inputs|record|playback]
p lain t ext - ch an n el[main|display|cursor|inputs|record|playback]
imag e- co mp ressio n =<compress>
jp eg - wan - co mp ressio n =<value>
z lib - g lz - wan - co mp ressio n =<value>
st reamin g - vid eo =[off|all|filter]
ag en t - mo u se=[on|off]
p layb ack- co mp ressio n =[on|off]
seamless- mig rat io =[on|off]
Chapt er 2 0 . qemu- kvm Whit elist
20.5. Net work opt ions
T AP ne t wo rk
- n et d ev t ap ,id=<id>][,<options>...]
The following options are supported (all use name=value format): ifname fd script downscript sndbuf vnet_hdr
321
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
vhost vhostfd vhostforce
20.6. Device opt ions
Ge ne ral de vice
- d evice <driver>[,<prop>[=<value>][,...]]
All drivers support following properties id bus
Following drivers are supported (with available properties):
p ci- assig n
host bootindex configfd addr rombar romfile multifunction
If the device has multiple functions, all of them need to be assigned to the same guest.
rt l8139
mac netdev bootindex addr
e1000
mac netdev bootindex addr
virt io - n et - p ci
ioeventfd
322
x-txburst tx
q xl
addr ram_size vram_size revision cmdlog ctrl_vq ctrl_rx ctrl_vlan ctrl_rx_extra mac netdev bootindex x-txtimer guest_ecn guest_ufo host_tso4 host_tso6 host_ecn host_ufo mrg_rxbuf status vectors indirect event_idx csum guest_csum gso guest_tso4 guest_tso6
Chapt er 2 0 . qemu- kvm Whit elist
323
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
addr
id e- d rive
unit drive physical_block_size bootindex ver wwn
virt io - b lk- p ci
class drive logical_block_size physical_block_size min_io_size opt_io_size bootindex ioeventfd vectors indirect_desc event_idx scsi addr
virt io - scsi- p ci - tech-preview in 6.3, supported since 6.4.
For Windows guests, Windows Server 2003, which was tech-preview, is no longer supported since 6.5. However, Windows Server 2008 and 2012, and Windows desktop 7 and 8 are fully supported since 6.5.
vectors indirect_desc event_idx num_queues addr
isa- d eb u g co n isa- serial
324
ES1370
addr
AC9 7
addr event_idx max_ports flow_control addr
in t el- h d a
addr
h d a- d u p lex
cad
h d a- micro
cad
h d a- o u t p u t
cad index iobase irq chardev
virt serialp o rt
nr chardev name
virt co n so le
nr chardev name
virt io - serial- p ci
vectors class indirect_desc
Chapt er 2 0 . qemu- kvm Whit elist
325
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide i6 300esb
addr
ib 700 - no properties
sg a - no properties
virt io - b allo o n - p ci
indirect_desc event_idx addr
u sb - t ab let
migrate port
u sb - kb d
migrate port
u sb - mo u se
migrate port
u sb - ccid - supported since 6.2
port slot
u sb - h o st - tech preview since 6.2
hostbus hostaddr hostport vendorid productid isobufs port
u sb - h u b - supported since 6.2
port
u sb - eh ci - tech preview since 6.2
freq
326
maxframes port
u sb - st o rag e - tech preview since 6.2
drive bootindex serial removable port
u sb - red ir - tech preview for 6.3, supported since 6.4
chardev filter
scsi- cd - tech preview for 6.3, supported since 6.4
drive logical_block_size physical_block_size min_io_size opt_io_size bootindex ver serial scsi-id lun channel-scsi wwn
scsi- h d -tech preview for 6.3, supported since 6.4
drive logical_block_size physical_block_size min_io_size opt_io_size bootindex ver
Chapt er 2 0 . qemu- kvm Whit elist
327
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
serial scsi-id lun channel-scsi wwn
scsi- b lo ck -tech preview for 6.3, supported since 6.4
drive bootindex
scsi- d isk -tech preview for 6.3
drive=drive logical_block_size physical_block_size min_io_size opt_io_size bootindex ver serial scsi-id lun channel-scsi wwn
p iix3- u sb - u h ci p iix4 - u sb - u h ci ccid - card - p asst h ru
Glo bal de vice se t t ing
- g lo b al <device>.<property>=<value>
Supported devices and properties as in "General device" section with these additional devices:
isa- f d c
driveA driveB bootindexA
328
Chapt er 2 0 . qemu- kvm Whit elist
bootindexB
q xl- vg a
ram_size vram_size revision cmdlog addr
Charact e r de vice
- ch ard ev backend,id=<id>[,<options>]
Supported backends are:
n u ll,id=<id> - null device
so cket ,id=<id>,port=<port>[,host=<host>][,to=<to>][,ipv4][,ipv6][,nodelay][,server][,nowait][,telnet]
- tcp socket
so cket ,id=<id>,path=<path>[,server][,nowait][,telnet] - unix socket
f ile,id=<id>,path=<path> - trafit to file.
st d io ,id=<id> - standard i/o
sp icevmc,id=<id>,name=<name> - spice channel
Enable USB
- u sb
20.7. Linux/Mult iboot boot
Ke rne l file
- kern el <bzImage>
Note: multiboot images are not supported
Ram disk
- in it rd <file>
Co m m and line param e t e r
- ap p en d <cmdline>
20.8. Expert opt ions
KVM virt ualizat io n
329
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
- en ab le- kvm
QEMU-KVM supports only KVM virtualization and it is used by default if available. If -enable-kvm is used and KVM is not available, qemu-kvm fails. However, if -enable-kvm is not used and KVM is not available, q emu - kvm runs in TCG mode, which is not supported.
Disable ke rne l m o de PIT re inje ct io n
- n o - kvm- p it - rein ject io n
No shut do wn
- n o - sh u t d o wn
No re bo o t
- n o - reb o o t
Se rial po rt , m o nit o r, QMP
- serial <dev>
- mo n it o r <dev>
- q mp <dev>
Supported devices are:
st d io - standard input/output
n u ll - null device
f ile:<filename> - output to file.
t cp :[<host>]:<port>[,server][,nowait][,nodelay] - TCP Net console.
u n ix:<path>[,server][,nowait] - Unix domain socket.
mo n :<dev_string> - Any device above, used to multiplex monitor too.
n o n e - disable, valid only for -serial.
ch ard ev:<id> - character device created with -chardev.
Mo nit o r re dire ct
- mo n <chardev_id>[,mode=[readline|control]][,default=[on|off]]
Manual CPU st art
- S
RT C
- rt c [base=utc|localtime|date][,clock=host|vm][,driftfix=none|slew]
Wat chdo g
330
Chapt er 2 0 . qemu- kvm Whit elist
- wat ch d o g model
Wat chdo g re act io n
- wat ch d o g - act io n <action>
Gue st m e m o ry backing
- mem- p reallo c - mem- p at h /dev/hugepages
SMBIOS e nt ry
- smb io s type=0[,vendor=<str>][,<version=str>][,date=<str>][,release=% d.% d]
- smb io s type=1[,manufacturer=<str>][,product=<str>][,version=<str>][,serial=<str>][,uuid=<uuid>]
[,sku=<str>][,family=<str>]
20.9. Help and informat ion opt ions
He lp
- h
- h elp
Ve rsio n
- versio n
Audio he lp
- au d io - h elp
20.10. Miscellaneous opt ions
Migrat io n
- in co min g
No de fault co nfigurat io n
- n o d ef co n f ig
- n o d ef au lt s
Running without -nodefaults is not supported
De vice co nfigurat io n file
- read co n f ig <file>
- writ eco n f ig <file>
331
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Lo ade d save d st at e
- lo ad vm <file>
332
Chapt er 2 1 . Manipulat ing t he domain xml
Chapter 21. Manipulating the domain xml
This section describes the XML format used to represent domains. Here the term domain refers to the root <d o mai n> element required for all guest virtual machine. The domain XML has two attributes:
type specifies the hypervisor used for running the domain. The allowed values are driver specific, but include KVM and others. i d is a unique integer identifier for the running guest virtual machine.
Inactive machines have no id value. The sections in this chapter will address the components of the domain XML. Additional chapters in this manual may refer to this chapter when manipulation of the domain XML is required.
21.1. General informat ion and met adat a
This information is in this part of the domain XML:
<domain type= 'xen' id= '3' >
<name> fv0 </name>
<uuid> 4dea22b31d52d8f32516782e98ab3fa0 </uuid>
<title> A short description - title - of the domain </title>
<description> Some human readable description </description>
<metadata>
<app1:foo xmlns:app1= "http://app1.org/app1/" > ..
</app1:foo>
<app2:bar xmlns:app2= "http://app1.org/app2/" > ..
</app2:bar>
</metadata>
...
</domain>
Fig u re 21.1. Do main XML met ad at a
The components of this section of the domain XML are as follows:
T ab le 21.1. G en eral met ad at a elemen t s
Elemen t
<name>
<uui d >
Descrip t io n
Assigns a name for the virtual machine. This name should consist only of alpha-numeric characters and is required to be unique within the scope of a single host physical machine. It is often used to form the filename for storing the persistent configuration files.
assigns a globally unique identifier for the virtual machine. The format must be RFC 4122 compliant, eg 3e3fce4 5-4 f53-4 fa7-bb32-
11f34 16 8b82b. If omitted when defining/creating a new machine, a random
UUID is generated. It is also possible to provide the UUID via a sysinfo specification.
333
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Elemen t
<ti tl e>
<d escri pti o n>
<metad ata>
Descrip t io n
ti tl e Creates space for a short description of the domain. The title should not contain any newlines.
Different from the title, This data is not used by libvirt in any way, it can contain any information the user wants to display.
Can be used by applications to store custom metadata in the form of XML nodes/trees.
Applications must use custom namespaces on their XML nodes/trees, with only one top-level element per namespace (if the application needs structure, they should have sub-elements to their namespace element)
21.2. Operat ing syst em boot ing
There are a number of different ways to boot virtual machines each with their own pros and cons.
Each one is described in the sub-sections that follow and include: BIOS bootloader, Host physical machine bootloader, direct kerel boot, and container boot.
21.2.1. BIOS boot loader
Booting via the BIOS is available for hypervisors supporting full virtualization. In this case the BIOS has a boot order priority (floppy, harddisk, cdrom, network) determining where to obtain/find the boot image. The OS section of the domain XML contains the information as follows:
...
<os>
<type> hvm </type>
<loader> /usr/lib/xen/boot/hvmloader </loader>
<boot dev= 'hd' />
<boot dev= 'cdrom' />
<bootmenu enable= 'yes' />
<smbios mode= 'sysinfo' />
<bios useserial= 'yes' rebootTimeout= '0' />
</os>
...
Fig u re 21.2. BO IS b o o t lo ad er d o main XML
The components of this section of the domain XML are as follows:
T ab le 21.2. BIO S b o o t lo ad er elemen t s
Elemen t Descrip t io n
334
Elemen t
<type>
<l o ad er>
<bo o t>
<bo o tmenu>
<smbi o s>
<bi o s>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
Specifies the type of operating system to be booted on the guest virtual machine. hvm indicates that the OS is one designed to run on bare metal, so requires full virtualization. l i nux refers to an OS that supports the Xen 3 hypervisor guest ABI. There are also two optional attributes, arch specifying the CPU architecture to virtualization, and machi ne referring to the machine type. Refer to
Driver
Capabilities
for more information.
refers to a piece of firmware that is used to assist the domain creation process. It is only needed for using Xen fully virtualized domains.
takes one of the values:fd , hd , cd ro m or
netwo rk and is used to specify the next boot device to consider. The boot element can be repeated multiple times to setup a priority list of boot devices to try in turn. Multiple devices of the same type are sorted according to their targets while preserving the order of buses. After defining the domain, its XML configuration returned by libvirt (through virDomainGetXMLDesc) lists devices in the sorted order. Once sorted, the first device is marked as bootable. For more information see
BIOS bootloader
.
determines whether or not to enable an interactive boot menu prompt on guest virtual machine startup. The enabl e attribute can be either yes or no . If not specified, the hypervisor default is used determines how SMBIOS information is made visible in the guest virtual machine. The mo d e attribute must be specified, as either emul ate
(lets the hypervisor generate all values),
ho st(copies all of Block 0 and Block 1, except for the UUID, from the host physical machine's
SMBIOS values; the virConnectGetSysinfo call can be used to see what values are copied), or
sysi nfo (uses the values in the sysinfo element). If not specified, the hypervisor default setting is used.
This element has attribute useseri al with possible values yes or no . The attribute enables or disables Serial Graphics Adapter which allows users to see BIOS messages on a serial port. Therefore, one needs to have serial port defined. Note there is another attribute,
rebo o tT i meo ut that controls whether and after how long the guest virtual machine should start booting again in case the boot fails
(according to BIOS). The value is in milliseconds with maximum of 6 5535 and special value -1 disables the reboot.
335
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
21.2.2. Host physical machine boot loader
Hypervisors employing paravirtualization do not usually emulate a BIOS, but instead the host physical machine is responsible for the operating system boot. This may use a pseudo-bootloader in the host physical machine to provide an interface to choose a kernel for the guest virtual machine. An example is pygrub with Xen.
...
<bootloader> /usr/bin/pygrub </bootloader>
<bootloader_args> --append single </bootloader_args>
...
Fig u re 21.3. Ho st p h ysical mach in e b o o t lo ad er d o main XML
The components of this section of the domain XML are as follows:
T ab le 21.3. BIO S b o o t lo ad er elemen t s
Elemen t
<bo o tl o ad er>
<bo o tl o ad er_arg s>
Descrip t io n
provides a fully qualified path to the bootloader executable in the host physical machine OS.
This bootloader will choose which kernel to boot. The required output of the bootloader is dependent on the hypervisor in use.
allows command line arguments to be passed to the bootloader (optional command)
21.2.3. Direct kernel boot
When installing a new guest virtual machine OS, it is often useful to boot directly from a kernel and initrd stored in the host physical machine OS, allowing command line arguments to be passed directly to the installer. This capability is usually available for both para and full virtualized guest virtual machines.
...
<os>
<type> hvm </type>
<loader> /usr/lib/xen/boot/hvmloader </loader>
<kernel> /root/f8-i386-vmlinuz </kernel>
<initrd> /root/f8-i386-initrd </initrd>
<cmdline> console=ttyS0 ks=http://example.com/f8-i386/os/ </cmdline>
<dtb> /root/ppc.dtb
</dtb>
</os>
...
336
Chapt er 2 1 . Manipulat ing t he domain xml
Fig u re 21.4 . Direct kern el b o o t
The components of this section of the domain XML are as follows:
T ab le 21.4 . Direct kern el b o o t elemen t s
Elemen t
<type>
<l o ad er>
<kernel >
<i ni trd >
<cmd l i ne>
Descrip t io n
same as described in the BIOS boot section same as described in the BIOS boot section specifies the fully-qualified path to the kernel image in the host physical machine OS specifies the fully-qualified path to the (optional) ramdisk image in the host physical machine OS.
specifies arguments to be passed to the kernel
(or installer) at boot time. This is often used to specify an alternate primary console (eg serial port), or the installation media source / kickstart file
21.2.4 . Cont ainer boot
When booting a domain using container based virtualization, instead of a kernel or boot image, a path to the init binary is required, using the init element. By default this will be launched with no arguments. To specify the initial argv, use the i ni targ element, repeated as many times as required.
The cmd l i ne element, provides an equivalent to /pro c/cmd l i ne but will not effect <i ni targ >.
>
...
<os>
<type> hvm </type>
<loader> /usr/lib/xen/boot/hvmloader </loader>
<kernel> /root/f8-i386-vmlinuz </kernel>
<initrd> /root/f8-i386-initrd </initrd>
<cmdline> console=ttyS0 ks=http://example.com/f8-i386/os/ </cmdline>
<dtb> /root/ppc.dtb
</dtb>
</os>
...
Fig u re 21.5. Co n t ain er b o o t
21.3. SMBIOS syst em informat ion
Some hypervisors allow control over what system information is presented to the guest virtual machine (for example, SMBIOS fields can be populated by a hypervisor and inspected via the dmidecode command in the guest virtual machine). The optional sysinfo element covers all such categories of information.
337
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
...
<os>
<smbios mode= 'sysinfo' />
...
</os>
<sysinfo type= 'smbios' >
<bios>
<entry name= 'vendor' > LENOVO </entry>
</bios>
<system>
<entry name= 'manufacturer' > Fedora </entry>
<entry name= 'vendor' > Virt-Manager </entry>
</system>
</sysinfo>
...
Fig u re 21.6 . SMBIO S syst em in f o rmat io n
The <sysi nfo > element has a mandatory attribute type that determines the layout of sub-elements, and may be defined as follows:
smbi o s - Sub-elements call out specific SMBIOS values, which will affect the guest virtual machine if used in conjunction with the smbios sub-element of the <o s> element. Each subelement of sysinfo names a SMBIOS block, and within those elements can be a list of entry elements that describe a field within the block. The following blocks and entries are recognized:
bi o s - This is block 0 of SMBIOS, with entry names drawn from vend o r, versi o n, d ate, and
rel ease.
<system> - This is block 1 of SMBIOS, with entry names drawn from manufacturer,
pro d uct, versi o n, seri al , uui d , sku, and fami l y. If a uui d entry is provided alongside a top-level uuid element, the two values must match.
21.4 . CPU allocat ion
<domain>
...
<vcpu placement= 'static' cpuset= "1-4,^3,6" current= "1" > 2 </vcpu>
...
</domain>
Fig u re 21.7. CPU allo cat io n
338
Chapt er 2 1 . Manipulat ing t he domain xml
The <cpu> element defines the maximum number of virtual CPUs allocated for the guest virtual machine OS, which must be between 1 and the maximum supported by the hypervisor. This element can contain an optional cpuset attribute, which is a comma-separated list of physical CPU numbers that domain process and virtual CPUs can be pinned to by default.
Note that the pinning policy of domain process and virtual CPUs can be specified separately by using the cputune attribute. If attribute emul ato rpi n of <cputune> is specified, cpuset specified by <vcpu> will be ingored.
Similarly, virtual CPUs that have set a value for vcpupi n cause cpuset settings to be ignored. For virtual CPUs where vcpupi n is not specified, it will be pinned to the physical CPUs specified by
cpuset. Each element in the cpuset list is either a single CPU number, a range of CPU numbers, or a caret (^) followed by a CPU number to be excluded from a previous range. The attribute current can be used to specify whether fewer than the maximum number of virtual CPUs should be enabled.
The optional attribute pl acement can be used to indicate the CPU placement mode for domain process, its value can be either stati c or auto , which defaults to pl acement, or numatune, or
stati c if cpuset is specified. auto indicates the domain process will be pinned to the advisory nodeset from querying numad, and the value of attribute cpuset will be ignored if it's specified. If both cpuset and pl acement are not specified, or if placement is stati c, but no cpuset is specified, the domain process will be pinned to all the available physical CPUs.
21.5. CPU t uning
<domain>
...
<cputune>
<vcpupin vcpu= "0" cpuset= "1-4,^2" />
<vcpupin vcpu= "1" cpuset= "0,1" />
<vcpupin vcpu= "2" cpuset= "2,3" />
<vcpupin vcpu= "3" cpuset= "0,4" />
<emulatorpin cpuset= "1-3" />
<shares> 2048 </shares>
<period> 1000000 </period>
<quota> -1 </quota>
<emulator_period> 1000000 </emulator_period>
<emulator_quota> -1 </emulator_quota>
</cputune>
...
</domain>
Fig u re 21.8. CPU t u n in g
Although all are optional, the components of this section of the domain XML are as follows:
T ab le 21.5. CPU t u n in g elemen t s
Elemen t
<cputune>
Descrip t io n
Provides details regarding the CPU tunable parameters for the domain. This is optional.
339
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Elemen t
<vcpupi n>
<emul ato rpi n>
<shares>
<peri o d >
<q uo ta>
<emul ato r_peri o d >
Descrip t io n
Specifies which of host physical machine's physical CPUs the domain VCPU will be pinned to. If this is omitted, and attribute cpuset of element <vcpu> is not specified, the vCPU is pinned to all the physical CPUs by default. It contains two required attributes, the attribute
vcpu specifies i d , and the attribute cpuset is same as attribute cpuset of element <vC P U>.
Specifies which of the host physical machine
CPUs, the "emulator", a subset of a domains not including vcpu, will be pinned to. If this is omitted, and attribute cpuset of element
<vcpu> is not specified, the "emulator" is pinned to all the physical CPUs by default. It contains one required attribute cpuset specifying which physical CPUs to pin to.
emul ato rpi n is not allowed if attribute
pl acement of element <vcpu> is auto .
Specifies the proportional weighted share for the domain. If this is omitted, it defaults to the OS provided defaults. If there is no unit for the value, itn is calculated relative to the setting of other guest virtual machine. For example, if a guest virtual machine is configured with value
2048 will get twice as much CPU time as a guest virtual machine configured with value 1024.
Specifies the enforcement intervalin microseconds. By using peri o d , each of the domain's vcpu will not be allowed to consume more than its allotted quota worth of run time.
This value should be within the following range:
10 0 0 -10 0 0 0 0 0 . A peri o d > with a value of 0 means no value.
Specifies the maximum allowed bandwidth in microseconds. A domain with q uo ta as any negative value indicates that the domain has infinite bandwidth, which means that it is not bandwidth controlled. The value should be within the following range:10 0 0 -
184 4 6 74 4 0 7370 9 551 or less than 0 . A q uo ta with value of 0 means no value. You can use this feature to ensure that all vcpus run at the same speed.
Specifies the enforcement interval in microseconds. Within an
<emul ato r_peri o d >, emulator threads (those excluding vcpus) of the domain will not be allowed to consume more than the
<emul ato r_q uo ta> worth of run time. The
<emul ato r_peri o d > value should be in the following range: 10 0 0 - 10 0 0 0 0 0 . An
<emul ato r_peri o d > with value of 0 , means no value.
34 0
Elemen t
<emul ato r_q uo ta>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
Specifies the maximum allowed bandwidth in microseconds for the domain's emulator threads
(those excluding vcpus). A domain with an
<emul ato r_q uo ta> as a negative value indicates that the domain has infinite bandwidth for emulator threads (those excluding vcpus), which means that it is not bandwidth controlled.
The value should be in the following range:
10 0 0 - 184 4 6 74 4 0 7370 9 551, or less than
0 . An <emul ato r_q uo ta> with value 0 means no value.
21.6. Memory backing
Memory backing allows the hypervisor to properly manage large pages within the guest virtual machine. Once confured the following domain XML is effected:
<domain>
...
<memoryBacking>
<hugepages/>
</memoryBacking>
...
</domain>
Fig u re 21.9 . Memo ry b ackin g
The optional <memo ryBacki ng > element, may have an <hug epag es> element set within it. This tells the hypervisor that the guest virtual machine should have its memory allocated using hugepages instead of the normal native page size.
21.7. Memory t uning
<domain>
...
<memtune>
<hard_limit unit= 'G' > 1 </hard_limit>
<soft_limit unit= 'M' > 128 </soft_limit>
<swap_hard_limit unit= 'G' > 2 </swap_hard_limit>
<min_guarantee unit= 'bytes' > 67108864 </min_guarantee>
</memtune>
...
</domain>
34 1
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 21.10. Memo ry t u n in g
Although all are optional, the components of this section of the domain XML are as follows:
T ab le 21.6 . Memo ry t u n in g elemen t s
Elemen t
<memtune>
<hard _l i mi t>
<so ft_l i mi t>
<swap_hard _l i mi t>
<mi n_g uarantee>
Descrip t io n
Provides details regarding the memory tunable parameters for the domain. If this is omitted, it defaults to the OS provided defaults. The parameters are applied to the process as a whole therefore when setting limits, one needs to add up guest virtual machine RAM, guest virtual machine video RAM, and allow for some memory overhead. The last piece is hard to determine so one use trial and error. For each tunable, it is possible to designate which unit the number is in on input, using the same values as for
<memo ry>. For backwards compatibility, output is always in KiB.
This is the maximum memory the guest virtual machine can use. The uni t for this value is expressed in ki bi bytes (i.e. blocks of 1024 bytes)
This is the memory limit to enforce during memory contention. The uni t for this value is expressed in kibibytes (i.e. blocks of 1024 bytes)
This is the maximum memory plus swap the guest virtual machine can use. The uni t for this value is expressed in kibibytes (i.e. blocks of
1024 bytes). This has to be more than
<hard _l i mi t> value provided
This is the guaranteed minimum memory allocation for the guest virtual machine. The units for this value is expressed in kibibytes (i.e.
blocks of 1024 bytes)
21.8. NUMA node t uning
Once NUMA node tuning is done using conventional management tools the following domain XML parameters are effected:
>
<domain>
...
<numatune>
<memory mode= "strict" nodeset= "1-4,^3" />
</numatune>
...
</domain>
34 2
Chapt er 2 1 . Manipulat ing t he domain xml
Fig u re 21.11. NUMA n o d e t u n in g
Although all are optional, the components of this section of the domain XML are as follows:
T ab le 21.7. NUMA n o d e t u n in g elemen t s
Elemen t
<numatune>
<memo ry>
Descrip t io n
Provides details of how to tune the performance of a NUMA host physical machine via controlling NUMA policy for domain process.
Specifies how to allocate memory for the domain process on a NUMA host physical machine. It contains several optional attributes. Attribute
mo d e is either i nterl eave, stri ct, or
preferred . If no value is given it defaults to
stri ct. Attribute no d eset specifies the NUMA nodes, using the same syntax as attribute
cpuset of element <vcpu>. Attribute
pl acement can be used to indicate the memory placement mode for the domain process. Its value can be either stati c or auto . If attribute
<no d eset> is specified it defaults to the
<pl acement> of <vcpu>, or stati c. auto indicates the domain process will only allocate memory from the advisory nodeset returned from querying numad and the value of attribute nodeset will be ignored if it's specified. If attribute pl acement of vcpu is auto , and attribute <numatune> is not specified, a default numatune with <pl acement> auto and mode
stri ct will be added implicitly.
21.9. Block I/O t uning
<domain>
...
<blkiotune>
<weight> 800 </weight>
<device>
<path> /dev/sda </path>
<weight> 1000 </weight>
</device>
<device>
<path> /dev/sdb </path>
<weight> 500 </weight>
</device>
</blkiotune>
...
</domain>
34 3
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Fig u re 21.12. Blo ck I/O t u n in g
Although all are optional, the components of this section of the domain XML are as follows:
T ab le 21.8. Blo ck I/O t u n in g elemen t s
Elemen t
<bl ki o tune>
<wei g ht>
<d evi ce>
Descrip t io n
This optional element provides the ability to tune
Blkio cgroup tunable parameters for the domain.
If this is omitted, it defaults to the OS provided defaults.
This optional weight element is the overall I/O weight of the guest virtual machine. The value should be within the range 100 - 1000.
The domain may have multiple <d evi ce> elements that further tune the weights for each host physical machine block device in use by the domain. Note that multiple guest virtual machine disks can share a single host physical machine block device. In addition, as they are backed by files within the same host physical machine file system, this tuning parameter is at the global domain level, rather than being associated with each guest virtual machine disk device (contrast this to the <i o tune> element which can be applied to a single <d i sk>). Each device element has two mandatory subelements, <path> describing the absolute path of the device, and <wei g ht> giving the relative weight of that device, which has an acceptable range of 100 - 1000.
21.10. Resource part it ioning
Hypervisors may allow for virtual machines to be placed into resource partitions, potentially with nesting of said partitions. The <reso urce> element groups together configuration related to resource partitioning. It currently supports a child element partition whose content defines the path of the resource partition in which to place the domain. If no partition is listed, then the domain will be placed in a default partition. It is the responsibility of the app/admin to ensure that the partition exists prior to starting the guest virtual machine. Only the (hypervisor specific) default partition can be assumed to exist by default.
<resource>
<partition> /virtualmachines/production </partition>
</resource>
Fig u re 21.13. Reso u rce p art it io n in g
Resource partitions are currently supported by the QEMU and LXC drivers, which map partition paths to cgroups directories in all mounted controllers.
34 4
Chapt er 2 1 . Manipulat ing t he domain xml
21.11. CPU model and t opology
This section covers the requirements for CPU model. Note that every hypervisor has its own policy for which CPU features guest will see by default. The set of CPU features presented to the guest by
QEMU/KVM depends on the CPU model chosen in the guest virtual machine configuration. q emu32 and q emu6 4 are basic CPU models but there are other models (with additional features) available.
Each model and its topology is specified using the following elements from the domain XML:
<cpu match= 'exact' >
<model fallback= 'allow' > core2duo </model>
<vendor> Intel </vendor>
<topology sockets= '1' cores= '2' threads= '1' />
<feature policy= 'disable' name= 'lahf_lm' />
</cpu>
Fig u re 21.14 . CPU mo d el an d t o p o lo g y examp le 1
<cpu mode= 'host-model' >
<model fallback= 'forbid' />
<topology sockets= '1' cores= '2' threads= '1' />
</cpu>
Fig u re 21.15. CPU mo d el an d t o p o lo g y examp le 2
<cpu mode= 'host-passthrough' />
Fig u re 21.16 . CPU mo d el an d t o p o lo g y examp le 3
In cases where no restrictions are to be put on either the CPU model nor its features, a simpler cpu element such as the following may be used.
<cpu>
<topology sockets= '1' cores= '2' threads= '1' />
</cpu>
Fig u re 21.17. CPU mo d el an d t o p o lo g y examp le 4
The components of this section of the domain XML are as follows:
34 5
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
T ab le 21.9 . CPU mo d el an d t o p o lo g y elemen t s
Elemen t
<cpu>
<match>
Descrip t io n
This is the main container for describing guest virtual machine CPU requirements.
Specifies how the virtual CPU is provided to the guest virtual machine match for these requirements. The match attribute can be omitted if topology is the only element within
<cpu>. Possible values for the match attribute are:
mi ni mum - the specified CPU model and features describes the minimum requested
CPU.
exact - the virtual CPU provided to the guest virtual machine will exactly match the specification
stri ct - the guest virtual machine will not be created unless the host physical machine
CPU exactly matches the specification.
Note that the match attribute can be omitted and will default to exact.
34 6
Elemen t
<mo d e>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
This optional attribute may be used to make it easier to configure a guest virtual machine CPU to be as close to the host physical machine CPU as possible. Possible values for the mode attribute are:
custo m - describes how the CPU is presented to the guest virtual machine. This is the default setting when the mo d e attribute is not specified. This mode makes it so that a persistent guest virtual machine will see the same hardware no matter what host physical machine the guest virtual machine is booted on.
ho st-mo d el - this is essentially a shortcut to copying host physical machine CPU definition from the capabilities XML into the domain XML. As the CPU definition is copied just before starting a domain, the same XML can be used on different host physical machines while still providing the best guest virtual machine CPU each host physical machine supports. Neither the match attribute nor any feature elements can be used in this mode. For more information see libvirt domain XML CPU models
ho st-passthro ug h With this mode, the
CPU visible to the guest virtual machine is exactly the same as the host physical machine CPU including elements that cause errors within libvirt. The obvious the downside of this mode is that the guest virtual machine environment cannot be reproduced on different hardware and therefore this mode is recommended with great caution. Neither mo d el nor feature elements are allowed in this mode.
Note that in both ho st-mo d el and ho st-
passthro ug h mode, the real (approximate in host-passthrough mode) CPU definition which would be used on current host physical machine can be determined by specifying VIR_DOMAIN_XML_UPDATE_CPU flag when calling virDomainGetXMLDesc
API. When running a guest virtual machine that might be prone to operating system reactivation when presented with different hardware, and which will be migrated between host physical machines with different capabilities, you can use this output to rewrite XML to the custom mode for more robust migration.
34 7
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Elemen t
<mo d el >
<vend o r>
<to po l o g y>
Descrip t io n
Specifies CPU model requested by the guest virtual machine. The list of available CPU models and their definition can be found in
cpu_map. xml file installed in libvirt's data directory. If a hypervisor is not able to use the exact CPU model, libvirt automatically falls back to a closest model supported by the hypervisor while maintaining the list of CPU features. An optional fal l back attribute can be used to forbid this behavior, in which case an attempt to start a domain requesting an unsupported CPU model will fail. Supported values for fallback attribute are: al l o w (this is the default), and
fo rbi d . The optional vend o r_i d attribute can be used to set the vendor id seen by the guest virtual machine. It must be exactly 12 characters long. If not set, the vendor id of the host physical machine is used. Typical possible values are
Authenti cAMD and G enui neIntel .
Specifies CPU vendor requested by the guest virtual machine. If this element is missing, the guest virtual machine runs on a CPU matching given features regardless of its vendor. The list of supported vendors can be found in
cpu_map. xml .
Specifies requested topology of virtual CPU provided to the guest virtual machine. Three non-zero values have to be given for sockets, cores, and threads: total number of CPU sockets, number of cores per socket, and number of threads per core, respectively.
34 8
Elemen t
<feature>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
Can contain zero or more elements used to finetune features provided by the selected CPU model. The list of known feature names can be found in the same file as CPU models. The meaning of each feature element depends on its policy attribute, which has to be set to one of the following values:
fo rce - forces the virtual to be supported regardless of whether it is actually supported by host physical machine CPU.
req ui re - dictates that guest virtual machine creation will fail unless the feature is supported by host physical machine CPU.
This is the default setting
o pti o nal - this feature is supported by virtual CPU but and only if it is supported by host physical machine CPU.
d i sabl e - this is not supported by virtual
CPU.
fo rbi d - guest virtual machine creation will fail if the feature is supported by host physical machine CPU.
21.11.1. Guest virt ual machine NUMA t opology
Guest virtual machine NUMA topology can be specified using the <numa> element and the following from the domain XML:
<cpu>
<numa>
<cell cpus= '0-3' memory= '512000' />
<cell cpus= '4-7' memory= '512000' />
</numa>
</cpu>
...
Fig u re 21.18. G u est virt u al mach in e NUMA t o p o lo g y
Each cell element specifies a NUMA cell or a NUMA node. cpus specifies the CPU or range of CPUs that are part of the node. memo ry specifies the node memory in kibibytes (i.e. blocks of 1024 bytes).
Each cell or node is assigned cel l i d or no d ei d in increasing order starting from 0.
21.12. Event s configurat ion
34 9
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Using the following sections of domain XML it is possible to override the default actions taken on various events.
<on_poweroff> destroy </on_poweroff>
<on_reboot> restart </on_reboot>
<on_crash> restart </on_crash>
<on_lockfailure> poweroff </on_lockfailure>
Fig u re 21.19 . Even t s Co n f ig u rat io n
The following collections of elements allow the actions to be specified when a guest virtual machine
OS triggers a life cycle operation. A common use case is to force a reboot to be treated as a poweroff when doing the initial OS installation. This allows the VM to be re-configured for the first post-install bootup.
The components of this section of the domain XML are as follows:
T ab le 21.10. Even t co n f ig u rat io n elemen t s
St at e
<o n_po wero ff>
Descrip t io n
Specifies the action that is to be executed when the guest virtual machine requests a poweroff.
Four possible arguements are possible:
d estro y - this action terminates the domain completely and releases all resources
restart - this action terminates the domain completely and restarts it with the same configuration
preserve - this action terminates the domain completely but and its resources are preserved to allow for future analysis.
rename-restart - this action terminates the domain completely and then restarts it with a new name
350
St at e
<o n_rebo o t>
<o n_crash>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
Specifies the action that is to be executed when the guest virtual machine requests a reboot.Four
possible arguements are possible:
d estro y - this action terminates the domain completely and releases all resources
restart - this action terminates the domain completely and restarts it with the same configuration
preserve - this action terminates the domain completely but and its resources are preserved to allow for future analysis.
rename-restart - this action terminates the domain completely and then restarts it with a new name
Specifies the action that is to be executed when the guest virtual machine crashes. In addition, it supports these additional actions:
co red ump-d estro y - the crashed domain's core is dumped, domain is terminated completely, and all resources are released.
co red ump-restart - the crashed domain's core is dumped, and the domain is restarted with the same configuration settings
Four possible arguements are possible:
d estro y - this action terminates the domain completely and releases all resources
restart - this action terminates the domain completely and restarts it with the same configuration
preserve - this action terminates the domain completely but and its resources are preserved to allow for future analysis.
rename-restart - this action terminates the domain completely and then restarts it with a new name
351
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
St at e
<o n_l o ckfai l ure>
Descrip t io n
Specifies what action should be taken when a lock manager loses resource locks. The following actions are recognized by libvirt, although not all of them need to be supported by individual lock managers. When no action is specified, each lock manager will take its default action. The following arguments are possible:
po wero ff - forcefully powers off the domain
restart - restarts the domain to reacquire its locks.
pause - pauses the domain so that it can be manually resumed when lock issues are solved.
i g no re - keeps the domain running as if nothing happened.
21.13. Power Management
It is possible to forcibly enable or disable BIOS advertisements to the guest virtual machine OS using conventional management tools which effects the following section of the domain XML:
...
<pm>
<suspend-to-disk enabled= 'no' />
<suspend-to-mem enabled= 'yes' />
</pm>
...
Fig u re 21.20. Po wer Man ag emen t
The <pm> element can be enabled using the arguement yes or disabled using the argument no .
BIOS support can be implemented for S3 using the argument suspend -to -d i sk and S4 using the argument suspend -to -mem ACPI sleep states. If nothing is specified, the hypervisor will be left with its default value.
21.14 . Hypervisor feat ures
Hypervisors may allow certain CPU / machine features to be enabled (state= ' o n' ) or disabled
(state= ' o ff' ).
...
<features>
352
Chapt er 2 1 . Manipulat ing t he domain xml
<pae/>
<acpi/>
<apic/>
<hap/>
<privnet/>
<hyperv>
<relaxed state= 'on' />
</hyperv>
</features>
...
Fig u re 21.21. Hyp erviso r f eat u res
All features are listed within the <features> element, if a <state> is not specified it is disabled. The available features can be found by calling the capabi l i ti es XML, but a common set for fully virtualized domains are:
T ab le 21.11. Hyp erviso r f eat u res elemen t s
St at e
<pae>
<acpi >
<api c>
<hap> hyperv
Descrip t io n
Physical address extension mode allows 32-bit guest virtual machines to address more than 4
GB of memory.
Useful for power management, for example, with
KVM guest virtual machines it is required for graceful shutdown to work.
Allows the use of programmable IRQ management. For this element, there is an optional attribute eo i with values o n and o ff which sets the availability of EOI (End of
Interrupt) for the guest virtual machine.
Enables the use of Hardware Assisted Paging if it is available in the hardware.
Enables various features to improve the behavior of guest virtual machines running
Microsoft Windows. Using the optional attribute
rel axed with values o n or o ff enables or disables the relax constraints on timers
21.15. T ime keeping
The guest virtual machine clock is typically initialized from the host physical machine clock. Most operating systems expect the hardware clock to be kept in UTC, which is the default setting. Note that for Windows guest virtual machines the guest virtual machine must be set in l o cal ti me.
...
<clock offset= 'localtime' >
<timer name= 'rtc' tickpolicy= 'catchup' track= 'guest' >
353
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<catchup threshold= '123' slew= '120' limit= '10000' />
</timer>
<timer name= 'pit' tickpolicy= 'delay' />
</clock>
...
Fig u re 21.22. T ime keep in g
The components of this section of the domain XML are as follows:
T ab le 21.12. T ime keep in g elemen t s
St at e
<cl o ck>
<ti mer>
Descrip t io n
The o ffset attribute takes four possible values, allowing for fine grained control over how the guest virtual machine clock is synchronized to the host physical machine. Note that hypervisors are not required to support all policies across all time sources
utc - Synchronizes the clock to UTC when booted. utc mode can be converted to
vari abl e mode, which can be controlled by using the adjustment attribute. If the value is
reset, the conversion is not done. A numeric value forces the conversion to vari abl e mode using the value as the initial adjustment. The default adjustment is hypervisor specific.
l o cal ti me - Synchronizes the guest virtual machine clock with the host physical machine's configured timezone when booted.
The adjustment attribute behaves the same as in 'utc' mode.
ti mezo ne - Synchronizes the guest virtual machine clock to the requested timezone using the timezone attribute.
vari abl e - Gives the guest virtual machine clock an arbitrary offset applied relative to
UTC or localtime, depending on the basis attribute. The delta relative to UTC (or localtime) is specified in seconds, using the
ad justment attribute. The guest virtual machine is free to adjust the RTC over time and expect that it will be honored at next reboot. This is in contrast to utc and
l o cal ti me mode (with the optional attribute
ad justment= ' reset' ), where the RTC adjustments are lost at each reboot. In addition the basi s attribute can be either
utc (default) or l o cal ti me. The cl o ck element may have zero or more <ti mer> elements.
See Note
354
St at e
<freq uency>
<mo d e>
<present>
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
This is an unsigned integer specifying the frequency at which name="tsc" runs.
The mo d e attribute controls how the
name= "tsc" <ti mer> is managed, and can be set to: auto , nati ve, emul ate, paravi rt, or
smpsafe. Other timers are always emulated.
Specifies whether a particular timer is available to the guest virtual machine. Can be set to yes or no
Additional information about the
<ti mer>
element
Each <ti mer> element must contain a name attribute, and may have the following attributes depending on the name specified.
<name> - selects which ti mer is being modified. The following values are acceptable:kvmcl o ck (QEMU-KVM), pi t(QEMU-KVM), or rtc(QEMU-KVM), or tsc(libxl only). Note that pl atfo rm is currently unsupported.
track - specifies the timer track. The following values are acceptable: bo o t, g uest, or
wal l . track is only valid for name= "rtc".
ti ckpo l i cy - determines what happens whens the deadline for injecting a tick to the guest virtual machine is missed. The following values can be assigned:
d el ay -will continue to deliver ticks at the normal rate. The guest virtual machine time will be delayed due to the late tick
catchup - delivers ticks at a higher rate in order to catch up with the missed tick. The guest virtual machine time is not displayed once catchup is complete. In addition, there can be three optional attributes, each a positive integer, as follows: threshold, slew, and limit.
merg e - merges the missed tick(s) into one tick and injects them. The guest virtual machine time may be delayed, depending on how the merge is done.
d i scard - throws away the missed tick(s) and continues with future injection at its default interval setting. The guest virtual machine time may be delayed, unless there is an explicit statement for handling lost ticks
21.16. Devices
This set of XML elements are all used to describe devices provided to the guest virtual machine domain. All of the devices below are indicated as children of the main devices element.
...
<devices>
<emulator> /usr/lib/xen/bin/qemu-dm </emulator>
</devices>
...
Fig u re 21.23. Devices - ch ild elemen t s
355
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The contents of the <emul ato r> element specify the fully qualified path to the device model emulator binary. The capabilities XML specifies the recommended default emulator to use for each particular domain type or architecture combination.
21.16.1. Hard drives, floppy disks, CDROMs
This section of the domain XML specifies any device that looks like a disk, be it a floppy, harddisk, cdrom, or paravirtualized driver is specified via the disk element.
...
<devices>
<disk type= 'file' snapshot= 'external' >
<driver name= "tap" type= "aio" cache= "default" />
<source file= '/var/lib/xen/images/fv0' startupPolicy= 'optional' >
<seclabel relabel= 'no' />
</source>
<target dev= 'hda' bus= 'ide' />
<iotune>
<total_bytes_sec> 10000000 </total_bytes_sec>
<read_iops_sec> 400000 </read_iops_sec>
<write_iops_sec> 100000 </write_iops_sec>
</iotune>
<boot order= '2' />
<encryption type= '...' >
...
</encryption>
<shareable/>
<serial>
...
</serial>
</disk>
...
<disk type= 'network' >
<driver name= "qemu" type= "raw" io= "threads" ioeventfd= "on" event_idx= "off" />
<source protocol= "sheepdog" name= "image_name" >
<host name= "hostname" port= "7000" />
</source>
<target dev= "hdb" bus= "ide" />
<boot order= '1' />
<transient/>
<address type= 'drive' controller= '0' bus= '1' unit= '0' />
</disk>
<disk type= 'network' >
<driver name= "qemu" type= "raw" />
<source protocol= "rbd" name= "image_name2" >
<host name= "hostname" port= "7000" />
</source>
<target dev= "hdd" bus= "ide" />
<auth username= 'myuser' >
<secret type= 'ceph' usage= 'mypassid' />
</auth>
356
Chapt er 2 1 . Manipulat ing t he domain xml
</disk>
<disk type= 'block' device= 'cdrom' >
<driver name= 'qemu' type= 'raw' />
<target dev= 'hdc' bus= 'ide' tray= 'open' />
<readonly/>
</disk>
<disk type= 'block' device= 'lun' >
<driver name= 'qemu' type= 'raw' />
<source dev= '/dev/sda' />
<target dev= 'sda' bus= 'scsi' />
<address type= 'drive' controller= '0' bus= '0' target= '3' unit= '0' />
</disk>
<disk type= 'block' device= 'disk' >
<driver name= 'qemu' type= 'raw' />
<source dev= '/dev/sda' />
<geometry cyls= '16383' heads= '16' secs= '63' trans= 'lba' />
<blockio logical_block_size= '512' physical_block_size= '4096' />
<target dev= 'hda' bus= 'ide' />
</disk>
<disk type= 'volume' device= 'disk' >
<driver name= 'qemu' type= 'raw' />
<source pool= 'blk-pool0' volume= 'blk-pool0-vol0' />
<target dev= 'hda' bus= 'ide' />
</disk>
</devices>
...
Fig u re 21.24 . Devices - Hard d rives, f lo p p y d isks, CDRO Ms
2 1 .1 6 .1 .1 . Disk e le m e nt
The <d i sk> element is the main container for describing disks. The attribute type can be used with the <d i sk> element. The following types are allowed:
fi l e bl o ck d i r netwo rk
For more information, see Disk Elements
2 1 .1 6 .1 .2 . So urce e le m e nt
If the <d i sk type= ' fi l e' ' >, then the fi l e attribute specifies the fully-qualified path to the file holding the disk. If the <d i sk type= ' bl o ck' >, then the d ev attribute specifies the path to the host physical machine device to serve as the disk. With both fi l e and bl o ck, one or more optional subelements secl abel , described below, can be used to override the domain security labeling policy for just that source file. If the disk type is d i r, then the d i r attribute specifies the fully-qualified path to the directory to use as the disk. If the disk type is netwo rk, then the protocol attribute specifies the protocol to access to the requested image; possible values are nbd , rbd , sheepd o g or g l uster.
If the protocol attribute is rbd , sheepd o g or g l uster, an additional attribute name is mandatory to specify which volume and or image will be used. When the disk type is netwo rk, the so urce may
357
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
have zero or more ho st sub-elements used to specify the host physical machines to connect, including: type= ' d i r' and type= ' netwo rk' . For a fi l e disk type which represents a cdrom or floppy (the device attribute), it is possible to define policy what to do with the disk if the source file is not accessible. This is done by manipulating the startupP o l i cy attribute, with the following values:
mand ato ry causes a failure if missing for any reason. This is the default setting.
req ui si te causes a failure if missing on boot up, drops if missing on migrate/restore/revert
o pti o nal drops if missing at any start attempt
2 1 .1 6 .1 .3. Mirro r e le m e nt
This element is present if the hypervisor has started a Bl o ckC o py operation, where the <mi rro r> location in the attribute file will eventually have the same contents as the source, and with the file format in attribute format (which might differ from the format of the source). If an attribute ready is present, then it is known the disk is ready to pivot; otherwise, the disk is probably still copying. For now, this element only valid in output; it is ignored on input.
2 1 .1 6 .1 .4 . T arge t e le m e nt
The <targ et> element controls the bus / device under which the disk is exposed to the guest virtual machine OS. The dev attribute indicates the logical device name. The actual device name specified is not guaranteed to map to the device name in the guest virtual machine OS. The optional bus attribute specifies the type of disk device to emulate; possible values are driver specific, with typical values being i d e, scsi , vi rti o , xen, usb or sata. If omitted, the bus type is inferred from the style of the device name. eg, a device named ' sd a' will typically be exported using a SCSI bus. The optional attribute tray indicates the tray status of the removable disks (i.e. CDROM or Floppy disk), the value can be either o pen or cl o sed . The default setting is cl o sed . For more information, see target
Elements
2 1 .1 6 .1 .5 . io t une
The optional <i o tune> element provides the ability to provide additional per-device I/O tuning, with values that can vary for each device (contrast this to the bl ki o tune element, which applies globally to the domain). This element has the following optional sub-elements. Note that any sub-element not specified or at all or specified with a value of 0 implies no limit.
<to tal _bytes_sec> - the total throughput limit in bytes per second. This element cannot be used with <read _bytes_sec> or <wri te_bytes_sec>.
<read _bytes_sec> - the read throughput limit in bytes per second.
<wri te_bytes_sec> - the write throughput limit in bytes per second.
<to tal _i o ps_sec> - the total I/O operations per second. This element cannot be used with
<read _i o ps_sec> or <wri te_i o ps_sec>.
<read _i o ps_sec> - the read I/O operations per second.
<wri te_i o ps_sec> - the write I/O operations per second.
2 1 .1 6 .1 .6 . drive r
The optional <d ri ver> element allows specifying further details related to the hypervisor driver that is used to provide the disk. The following options may be used:
358
Chapt er 2 1 . Manipulat ing t he domain xml
If the hypervisor supports multiple backend drivers, then the name attribute selects the primary backend driver name, while the optional type attribute provides the sub-type. For a list of possible types refer to Driver Elements
The optional cache attribute controls the cache mechanism, possible values are: d efaul t,
no ne, wri tethro ug h, wri teback, d i rectsync (similar to wri tethro ug h, but it bypasses the host physical machine page cache) and unsafe (host physical machine may cache all disk io, and sync requests from guest virtual machine virtual machines are ignored).
The optional erro r_po l i cy attribute controls how the hypervisor behaves on a disk read or write error, possible values are sto p, repo rt, i g no re, and eno space. The default setting of
erro r_po l i cy is repo rt. There is also an optional rerro r_po l i cy that controls behavior for read errors only. If no rerro r_po l i cy is given, erro r_po l i cy is used for both read and write errors. If rerro r_po l i cy is given, it overrides the erro r_po l i cy for read errors. Also note that
eno space is not a valid policy for read errors, so if erro r_po l i cy is set to eno space and no
rerro r_po l i cy is given, the read error the default setting, repo rt will be used.
The optional i o attribute controls specific policies on I/O; q emu guest virtual machine virtual machines support thread s and nati ve. The optional i o eventfd attribute allows users to set domain I/O asynchronous handling for disk device. The default is left to the discretion of the hypervisor. Accepted values are o n and o ff. Enabling this allows the guest virtual machine virtual machine to be executed while a separate thread handles I/O. Typically guest virtual machine virtual machines experiencing high system CPU utilization during I/O will benefit from this. On the other hand, an overloaded host physical machine can increase guest virtual machine virtual machine I/O latency. Unless you are absolutely certian that the i o needs to be manipulated, it is highly recommended that you not change the default setting and allow the hypervisor to dictate the setting.
The optional event_i d x attribute controls some aspects of device event processing and can be set to either o n or o ff - if it is on, it will reduce the number of interrupts and exits for the guest virtual machine virtual machine. The default is determined by the hypervisor and the default setting is o n. In cases that there is a situation where this behavior is suboptimal, this attribute provides a way to force the feature o ff. Unless you are absolutely certian that the event_i d x needs to be manipulated, it is highly recommended that you not change the default setting and allow the hypervisor to dictate the setting.
The optional co py_o n_read attribute controls whether to copy the read backing file into the image file. The accepted values can be either o n or <o ff>. co py-o n-read avoids accessing the same backing file sectors repeatedly and is useful when the backing file is over a slow network.
By default co py-o n-read is o ff.
2 1 .1 6 .1 .7 . Addit io nal De vice Ele m e nt s
The following attributes may be used within the d evi ce element:
<bo o t> - Specifies that the disk is bootable.
Ad d it io n al b o o t valu es
<o rd er> - Determines the order in which devices will be tried during boot sequence.
<per-d evi ce> boot elements cannot be used together with general boot elements in BIOS bootloader section
<encrypti o n> - Specifies how the volume is encrypted. See the Storage Encryption page for more information.
359
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<read o nl y> - Indicates the device cannot be modified by the guest virtual machine virtual machine. This setting is the default for disks with attri bute d evi ce= ' cd ro m' .
shareabl e Indicates the device is expected to be shared between domains (as long as hypervisor and OS support this). If shareabl e is used, cache= ' no ' should be used for that device.
<transi ent>- Indicates that changes to the device contents should be reverted automatically when the guest virtual machine virtual machine exits. With some hypervisors, marking a disk
transi ent prevents the domain from participating in migration or snapshots.
<seri al >- Specifies the serial number of guest virtual machine virtual machine virtual machine's hard drive. For example, <seri al >WD-WMAP9A966149</seri al >.
<wwn> - Specifies the WWN (World Wide Name) of a virtual hard disk or CD-ROM drive. It must be composed of 16 hexadecimal digits.
<vend o r> - Specifies the vendor of a virtual hard disk or CD-ROM device. It must not be longer than 8 printable characters.
<pro d uct> - Specifies the product of a virtual hard disk or CD-ROM device. It must not be longer than 16 printable characters
<ho st> - Supports 4 attributes: vi z, name, po rt, transpo rt and so cket, which specify the hostname, the port number, transport type and path to socket, respectively. The meaning of this element and the number of the elements depend on the pro to co l attribute as shown here:
ad d it io n al h o st at t rib u t es
nbd - Specifies a server running nbd-server and may only be used for only one host physical machine
rbd - Monitors servers of RBD type and may be used for one or more host physical machines
sheepd o g - Specifies one of the sheepdog servers (default is localhost:7000) and can be used one or none of the host physical machines
g l uster - Specifies a server running a glusterd daemon and may be used for only only one host physical machine. The valid values for transport attribute are tcp, rd ma or uni x. If nothing is specified, tcp is assumed. If transport is uni x, the so cket attribute specifies path to unix socket.
<ad d ress> - Ties the disk to a given slot of a controller. The actual <co ntro l l er> device can often be inferred by but it can also be explicitly specified. The type attribute is mandatory, and is typically pci or d ri ve. For a pci controller, additional attributes for bus, sl o t, and functi o n must be present, as well as optional d o mai n and mul ti functi o n. mul ti functi o n defaults to
o ff. For a d ri ve controller, additional attributes co ntro l l er, bus, targ et, and uni t are available, each with a default setting of 0 .
auth - Provides the authentication credentials needed to access the source. It includes a mandatory attribute username, which identifies the username to use during authentication, as well as a sub-element secret with mandatory attribute type. More information can be found here at
Device Elements
g eo metry - Provides the ability to override geometry settings. This mostly useful for S390 DASDdisks or older DOS-disks.
cyl s - Specifies the number of cylinders.
head s - Specifies the number of heads.
360
Chapt er 2 1 . Manipulat ing t he domain xml
secs - Specifies the number of sectors per track.
trans - Specifies the BIOS-Translation-Modus and can have the following values:no ne, l ba or
auto
bl o cki o - Allows the block device to be overridden with any of the block device properties listed below:
b lo ckio o p t io n s
l o g i cal _bl o ck_si ze- reports to the guest virtual machine virtual machine OS and describes the smallest units for disk I/O.
physi cal _bl o ck_si ze - reports to the guest virtual machine virtual machine OS and describes the disk's hardware sector size which can be relevant for the alignment of disk data.
21.16.2. Filesyst ems
A filesystems directory on the host physical machine that can be accessed directly from the guest virtual machine virtual machine
...
<devices>
<filesystem type= 'template' >
<source name= 'my-vm-template' />
<target dir= '/' />
</filesystem>
<filesystem type= 'mount' accessmode= 'passthrough' >
<driver type= 'path' wrpolicy= 'immediate' />
<source dir= '/export/to/guest' />
<target dir= '/import/from/host' />
<readonly/>
</filesystem>
...
</devices>
...
Fig u re 21.25. Devices - f ilesyst ems
The fi l esystem attribute has the following possible values:
type= ' mo unt' - Specifies the host physical machine directory to mount in the guest virtual machine. This is the default type if one is not specified. This mode also has an optional subelement d ri ver, with an attribute type= ' path' or type= ' hand l e' . The driver block has an optional attribute wrpo l i cy that further controls interaction with the host physical machine page cache; omitting the attribute reverts to the default setting, while specifying a value immediate means that a host physical machine writeback is immediately triggered for all pages touched during a guest virtual machine file write operation
type= ' templ ate' - Specifies the OpenVZ filesystem template and is only used by OpenVZ driver.
361
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
type= ' fi l e' - Specifies that a host physical machine file will be treated as an image and mounted in the guest virtual machine. This filesystem format will be autodetected and is only used by LXC driver.
type= ' bl o ck' - Specifies the host physical machine block device to mount in the guest virtual machine. The filesystem format will be autodetected and is only used by LXC driver.
type= ' ram' - Specifies that an in-memory filesystem, using memory from the host physical machine OS will be used. The source element has a single attribute usag e which gives the memory usage limit in kibibytes and is only used by LXC driver.
type= ' bi nd ' - Specifies a directory inside the guest virtual machine which will be bound to another directory inside the guest virtual machine. This element is only used by LXC driver.
accessmo d e which specifies the security mode for accessing the source. Currently this only works with type='mount' for the QEMU/KVM driver. The possible values are:
passthro ug h - Specifies that the source is accessed with the User's permission settings that are set from inside the guest virtual machine. This is the default accessmode if one is not specified.
mapped - Specifies that the source is accessed with the permission settings of the hypervisor.
sq uash - Similar to ' passthro ug h' , the exception is that failure of privileged operations like
cho wn are ignored. This makes a passthrough-like mode usable for people who run the hypervisor as non-root.
<so urce> - Specifies that the resource on the host physical machine that is being accessed in the guest virtual machine. The name attribute must be used with <type= ' templ ate' >, and the
d i r attribute must be used with <type= ' mo unt' >. The usag e attribute is used with
<type= ' ram' > to set the memory limit in KB.
targ et - Dictates where the source drivers can be accessed in the guest virtual machine. For most drivers this is an automatic mount point, but for QEMU-KVM this is merely an arbitrary string tag that is exported to the guest virtual machine as a hint for where to mount.
read o nl y - Enables exporting the filesytem as a readonly mount for guest virtual machine, by default read -wri te access is given.
space_hard _l i mi t - Specifies the maximum space available to this guest virtual machine's filesystem
space_so ft_l i mi t - Specifies the maximum space available to this guest virtual machine's filesystem. The container is permitted to exceed its soft limits for a grace period of time. Afterwards the hard limit is enforced.
21.16.3. Device addresses
Many devices have an optional <ad d ress> sub-element to describe where the device placed on the virtual bus is presented to the guest virtual machine. If an address (or any optional attribute within an address) is omitted on input, libvirt will generate an appropriate address; but an explicit address is required if more control over layout is required. See below for device examples including an address element.
Every address has a mandatory attribute type that describes which bus the device is on. The choice of which address to use for a given device is constrained in part by the device and the architecture of the guest virtual machine. For example, a disk device uses type= ' d i sk' , while a console device would use type= ' pci ' on i686 or x86_64 guest virtual machines, or type= ' spapr-vi o ' on
362
Chapt er 2 1 . Manipulat ing t he domain xml
PowerPC64 pseries guest virtual machines. Each address <type> has additional optional attributes that control where on the bus the device will be placed. The additional attributes are as follows:
type= ' pci ' - PCI addresses have the following additional attributes:
d o mai n (a 2-byte hex integer, not currently used by qemu)
bus (a hex value between 0 and 0xff, inclusive)
sl o t (a hex value between 0x0 and 0x1f, inclusive)
functi o n (a value between 0 and 7, inclusive)
Also available is the mul ti functi o n attribute, which controls turning on the multifunction bit for a particular slot/function in the PCI control register. This multifunction attribute defaults to
' o ff' , but should be set to ' o n' for function 0 of a slot that will have multiple functions used.
type= ' d ri ve - drive addresses have the following additional attributes:
co ntro l l er- (a 2-digit controller number)
bus - (a 2-digit bus number)
targ et - (a 2-digit bus number)
uni t - (a 2-digit unit number on the bus)
type= ' vi rti o -seri al ' - Each virtio-serial address has the following additional attributes:
co ntro l l er - (a 2-digit controller number)
bus - (a 2-digit bus number)
sl o t - (a 2-digit slot within the bus)
type= ' cci d ' - A CCID address, used for smart-cards, has the following additional attributes:
bus - (a 2-digit bus number)
sl o t attribute - (a 2-digit slot within the bus)
type= ' usb' - USB addresses have the following additional attributes:
bus - (a hex value between 0 and 0xfff, inclusive)
po rt - (a dotted notation of up to four octets, such as 1.2 or 2.1.3.1)
type= ' spapr-vi o - On PowerPC pseries guest virtual machines, devices can be assigned to the SPAPR-VIO bus. It has a flat 64-bit address space; by convention, devices are generally assigned at a non-zero multiple of 0x1000, but other addresses are valid and permitted by libvirt.
The additional attribute: reg (the hex value address of the starting register) can be assigned to this attribute.
21.16.4 . Cont rollers
Depending on the guest virtual machine architecture, it is possible to assign many virtual devices to a single bus. Under normal circumstances libvirt can automatically infer which controller to use for the bus. However, it may be necessary to provide an explicit <co ntro l l er> element in the guest virtual machine XML:
363
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
...
<devices>
<controller type= 'ide' index= '0' />
<controller type= 'virtio-serial' index= '0' ports= '16' vectors= '4' />
<controller type= 'virtio-serial' index= '1' >
<address type= 'pci' domain= '0x0000' bus= '0x00' slot= '0x0a' function= '0x0' />
<controller type= 'scsi' index= '0' model= 'virtio-scsi' num_queues= '8' />
</controller>
...
</devices>
...
Fig u re 21.26 . Co n t ro ller Elemen t s
Each controller has a mandatory attribute type, which must be one of "i d e", "fd c", "scsi ",
"sata", "usb", "cci d ", o r "vi rti o -seri al ", and a mandatory attribute i nd ex which is the decimal integer describing in which order the bus controller is encountered (for use in controller attributes of ad d ress elements). The "vi rti o -seri al " controller has two additional optional attributes, po rts and vecto rs, which control how many devices can be connected through the controller.
A <co ntro l l er type= ' scsi ' > has an optional attribute mo d el , which is one of "auto ",
"busl o g i c", "i bmvscsi ", "l si l o g i c", "l si as10 6 8", "vi rti o -scsi or "vmpvscsi ". It should be noted that virtio-scsi controllers and drivers will work on both KVM and Windows guest virtual machines. The <co ntro l l er type= ' scsi ' > also has an attribute num_q ueues which enables multi-queue support for the number of queues specified.
A "usb" controller has an optional attribute mo d el , which is one of "pi i x3-uhci ", "pi i x4 -
uhci ", "ehci ", "i ch9 -ehci 1", "i ch9 -uhci 1", "i ch9 -uhci 2", "i ch9 -uhci 3", "vt82c6 86 b-
uhci ", "pci -o hci " or "nec-xhci ". Additionally, if the USB bus needs to be explicitly disabled for the guest virtual machine, mo d el = ' no ne' may be used. The PowerPC64 "spapr-vio" addresses do not have an associated controller.
For controllers that are themselves devices on a PCI or USB bus, an optional sub-element ad d ress can specify the exact relationship of the controller to its master bus, with semantics given above.
USB companion controllers have an optional sub-element master to specify the exact relationship of the companion to its master controller. A companion controller is on the same bus as its master, so the companion index value should be equal.
...
<devices>
<controller type= 'usb' index= '0' model= 'ich9-ehci1' >
<address type= 'pci' domain= '0' bus= '0' slot= '4' function= '7' />
</controller>
364
Chapt er 2 1 . Manipulat ing t he domain xml
<controller type= 'usb' index= '0' model= 'ich9-uhci1' >
<master startport= '0' />
<address type= 'pci' domain= '0' bus= '0' slot= '4' function= '0' multifunction= 'on' />
</controller>
...
</devices>
...
Fig u re 21.27. Devices - co n t ro llers - USB
21.16.5. Device leases
When using a lock manager, you have the option to record device leases against a guest virtual machine. The lock manager will ensure that the guest virtual machine doesn't start unless the leases can be acquired. When configured using conventional management tools, the following section of the domain xml is effected:
...
<devices>
...
<lease>
<lockspace> somearea </lockspace>
<key> somekey </key>
<target path= '/some/lease/path' offset= '1024' />
</lease>
...
</devices>
...
Fig u re 21.28. Devices - d evice leases
The l ease section can have the following arguements:
l o ckspace - an arbitrary string that identifies lockspace within which the key is held. Lock managers may impose extra restrictions on the format, or length of the lockspace name.
key - an arbitrary string, that uniquely identies the lease to be acquired. Lock managers may impose extra restrictions on the format, or length of the key.
targ et - the fully qualified path of the file associated with the lockspace. The offset specifies where the lease is stored within the file. If the lock manager does not require a offset, set this value to 0 .
21.16.6. Host physical machine device assignment
2 1 .1 6 .6 .1 . USB / PCI de vice s
365
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The host physical machine's USB and PCI devices can be passed through to the guest virtual machine using the ho std ev element, by modifying the host physical machine using a management tool the following section of the domain xml file is configured:
...
<devices>
<hostdev mode= 'subsystem' type= 'usb' >
<source startupPolicy= 'optional' >
<vendor id= '0x1234' />
<product id= '0xbeef' />
</source>
<boot order= '2' />
</hostdev>
</devices>
...
Fig u re 21.29 . Devices - h o st p h ysical mach in e d evice assig n men t
Alternatively the following can also be done:
...
<devices>
<hostdev mode= 'subsystem' type= 'pci' managed= 'yes' >
<source>
<address bus= '0x06' slot= '0x02' function= '0x0' />
</source>
<boot order= '1' />
<rom bar= 'on' file= '/etc/fake/boot.bin' />
</hostdev>
</devices>
...
Fig u re 21.30. Devices - h o st p h ysical mach in e d evice assig n men t alt ern at ive
The components of this section of the domain XML are as follows:
T ab le 21.13. Ho st p h ysical mach in e d evice assig n men t elemen t s
Paramet er Descrip t io n
366
Paramet er ho std ev so urce vend o r, pro d uct bo o t
Chapt er 2 1 . Manipulat ing t he domain xml
Descrip t io n
This is the main container for describing host physical machine devices. For USB device passthrough mo d e is always subsystem and
type is usb for a USB device and pci for a PCI device. When manag ed is yes for a PCI device, it is detached from the host physical machine before being passed on to the guest virtual machine, and reattached to the host physical machine after the guest virtual machine exits. If
manag ed is omitted or no for PCI and for USB devices, the user is responsible to use the argument vi rNo d eD evi ceD ettach (or vi rsh
no d ed ev-d ettach) before starting the guest virtual machine or hot-plugging the device, and
vi rNo d eD evi ceR eAttach (or vi rsh
no d ed ev-reattach) after hot-unplug or stopping the guest virtual machine.
Describes the device as seen from the host physical machine. The USB device can either be addressed by vendor / product id using the
vend o r and pro d uct elements or by the device's address on the host physical machines using the ad d ress element. PCI devices on the other hand can only be described by their address. Note that the source element of USB devices may contain a startupP o l i cy attribute which can be used to define a rule for what to do if the specified host physical machine
USB device is not found. The attribute accepts the following values:
mand ato ry - fails if missing for any reason
(the default)
req ui si te - fails if missing on boot up, drops if missing on migrate/restore/revert
o pti o nal - drops if missing at any start attempt
These elements each have an i d attribute that specifies the USB vendor and product id. The
IDs can be given in decimal, hexadecimal
(starting with 0x) or octal (starting with 0) form.
Specifies that the device is bootable. The attribute's order determines the order in which devices will be tried during boot sequence. The per-device boot elements cannot be used together with general boot elements in BIOS bootloader section.
367
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er ro m ad d ress
Descrip t io n
Used to change how a PCI device's ROM is presented to the guest virtual machine. The optional bar attribute can be set to o n or o ff, and determines whether or not the device's ROM will be visible in the guest virtual machine's memory map. (In PCI documentation, the
ro mbar setting controls the presence of the
Base Address Register for the ROM). If no rom bar is specified, the default setting will be used.
The optional fi l e attribute is used to point to a binary file to be presented to the guest virtual machine as the device's ROM BIOS. This can be useful, for example, to provide a PXE boot ROM for a virtual function of an sr-iov capable ethernet device (which has no boot ROMs for the
VFs).
Also has a bus and d evi ce attribute to specify the USB bus and device number the device appears at on the host physical machine. The values of these attributes can be given in decimal, hexadecimal (starting with 0x) or octal
(starting with 0) form. For PCI devices the element carries 3 attributes allowing to designate the device as can be found with
l spci or with vi rsh no d ed ev-l i st
2 1 .1 6 .6 .2 . Blo ck / charact e r de vice s
The host physical machine's block / character devices can be passed through to the guest virtual machine by using management tools to modify the domain xml ho std ev element. Note that this is only possible with container based virtualization.
...
<hostdev mode= 'capabilities' type= 'storage' >
<source>
<block> /dev/sdf1 </block>
</source>
</hostdev>
...
Fig u re 21.31. Devices - h o st p h ysical mach in e d evice assig n men t b lo ck ch aract er d evices
An alternative approach is this:
368
Chapt er 2 1 . Manipulat ing t he domain xml
...
<hostdev mode= 'capabilities' type= 'misc' >
<source>
<char> /dev/input/event3 </char>
</source>
</hostdev>
...
Fig u re 21.32. Devices - h o st p h ysical mach in e d evice assig n men t b lo ck ch aract er d evices alt ern at ive 1
Another alternative approach is this:
...
<hostdev mode= 'capabilities' type= 'net' >
<source>
<interface> eth0 </interface>
</source>
</hostdev>
...
Fig u re 21.33. Devices - h o st p h ysical mach in e d evice assig n men t b lo ck ch aract er d evices alt ern at ive 2
The components of this section of the domain XML are as follows:
T ab le 21.14 . Blo ck / ch aract er d evice elemen t s
Paramet er ho std ev so urce
Descrip t io n
This is the main container for describing host physical machine devices. For block/character devices passthrough mo d e is always
capabi l i ti es and type is bl o ck for a block device and char for a character device.
This describes the device as seen from the host physical machine. For block devices, the path to the block device in the host physical machine
OS is provided in the nested bl o ck element, while for character devices the char element is used
21.16.7. Redirect ed devices
USB device redirection through a character device is supported by configuring it with management tools that modify the following section of the domain xml:
369
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
...
<devices>
<redirdev bus= 'usb' type= 'tcp' >
<source mode= 'connect' host= 'localhost' service= '4000' />
<boot order= '1' />
</redirdev>
<redirfilter>
<usbdev class= '0x08' vendor= '0x1234' product= '0xbeef' version= '2.00' allow= 'yes' />
<usbdev allow= 'no' />
</redirfilter>
</devices>
...
Fig u re 21.34 . Devices - red irect ed d evices
The components of this section of the domain XML are as follows:
T ab le 21.15. Red irect ed d evice elemen t s
Paramet er red i rd ev bo o t red i rfi l ter
Descrip t io n
This is the main container for describing redirected devices. bus must be usb for a USB device. An additional attribute type is required, matching one of the supported serial device types, to describe the host physical machine side of the tunnel; type= ' tcp' or
type= ' spi cevmc' (which uses the usbredir channel of a SPICE graphics device) are typical.
The redirdev element has an optional subelement ad d ress which can tie the device to a particular controller. Further sub-elements, such as so urce, may be required according to the given type, although atarg et sub-element is not required (since the consumer of the character device is the hypervisor itself, rather than a device visible in the guest virtual machine).
Specifies that the device is bootable. The order attribute determines the order in which devices will be tried during boot sequence. The perdevice boot elements cannot be used together with general boot elements in BIOS bootloader section.
This is used for creating the filter rule to filter out certain devices from redirection. It uses subelement usbd ev to define each filter rule. The
cl ass attribute is the USB Class code.
21.16.8. Smart card devices
370
Chapt er 2 1 . Manipulat ing t he domain xml
A virtual smartcard device can be supplied to the guest virtual machine via the smartcard element. A
USB smartcard reader device on the host physical machine cannot be used on a guest virtual machine with simple device passthrough, as it cannot be made available to both the host physical machine and guest virtual machine and can possibly lock the host physical machine computer when it is removed from the guest virtual machine. Therefore, some hypervisors provide a specialized virtual device that can present a smartcard interface to the guest virtual machine, with several modes for describing how the credentials are obtained from the host physical machine or even a from a channel created to a third-party smartcard provider. To set these parameters use a managment tool that will edit the following section of the domain XML:
USB device redirection through a character device is supported by configuring it with managment tools that modify the following section of the domain xml:
...
<devices>
<smartcard mode= 'host' />
<smartcard mode= 'host-certificates' >
<certificate> cert1 </certificate>
<certificate> cert2 </certificate>
<certificate> cert3 </certificate>
<database> /etc/pki/nssdb/ </database>
</smartcard>
<smartcard mode= 'passthrough' type= 'tcp' >
<source mode= 'bind' host= '127.0.0.1' service= '2001' />
<protocol type= 'raw' />
<address type= 'ccid' controller= '0' slot= '0' />
</smartcard>
<smartcard mode= 'passthrough' type= 'spicevmc' />
</devices>
...
Fig u re 21.35. Devices - smart card d evices
The smartcard element has a mandatory attribute mo d e. The following modes are supported; in each mode, the guest virtual machine sees a device on its USB bus that behaves like a physical USB
CCID (Chip/Smart Card Interface Device) card.
The mode attributes are as follows:
T ab le 21.16 . Smart card mo d e elemen t s
Paramet er mo d e= ' ho st'
Descrip t io n
In this mode, the hypervisor relays all requests from the guest virtual machine into direct access to the host physical machine's smartcard via
NSS. No other attributes or sub-elements are required. See below about the use of an optional ad d ress sub-element.
371
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er mo d e= ' ho st-certi fi cates' mo d e= ' passthro ug h'
Descrip t io n
This mode allows you to provide three NSS certificate names residing in a database on the host physical machine, rather than requiring a smartcard to be plugged into the host physical machine. These certificates can be generated via the command certuti l -d
/etc/pki /nssd b -x -t C T ,C T ,C T -S -s
C N= cert1 -n cert1, and the resulting three certificate names must be supplied as the content of each of three certi fi cate subelements. An additional sub-element d atabase can specify the absolute path to an alternate directory (matching the -d flag of the certuti l command when creating the certificates); if not present, it defaults to /etc/pki /nssd b.
Using this mode allows you to tunnel all requests through a secondary character device to a third-party provider (which may in turn be talking to a smartcard or using three certificate files, rather than having the hypervisor directly communicate with the host physical machine. In this mode of operation, an additional attribute
type is required, matching one of the supported serial device types, to describe the host physical machine side of the tunnel; type= ' tcp' or
type= ' spi cevmc' (which uses the smartcard channel of a SPICE graphics device) are typical.
Further sub-elements, such as so urce, may be required according to the given type, although a
targ et sub-element is not required (since the consumer of the character device is the hypervisor itself, rather than a device visible in the guest virtual machine).
Each mode supports an optional sub-element ad d ress, which fine-tunes the correlation between the smartcard and a ccid bus controller (Refer to
Section 21.16.3, “ Device addresses”
).
21.16.9. Net work int erfaces
The network interface devices are modified using management tools that will configure the following part of the Domain XML:
...
<devices>
<interface type= 'bridge' >
<source bridge= 'xenbr0' />
<mac address= '00:16:3e:5d:c7:9e' />
<script path= 'vif-bridge' />
<boot order= '1' />
372
Chapt er 2 1 . Manipulat ing t he domain xml
<rom bar= 'off' />
</interface>
</devices>
...
Fig u re 21.36 . Devices - n et wo rk in t erf aces
There are several possibilities for specifying a network interface visible to the guest virtual machine.
Each subsection below provides more details about common setup options. Additionally, each
<i nterface> element has an optional <ad d ress> sub-element that can tie the interface to a particular pci slot, with attribute type= ' pci ' (Refer to
Section 21.16.3, “ Device addresses”
).
2 1 .1 6 .9 .1 . Virt ual ne t wo rks
This is the recommended configuration for general guest virtual machine connectivity on host physical machines with dynamic / wireless networking configurations (or multi-host physical machine environments where the host physical machine hardware details are described separately in a <netwo rk> definition). In addition, it provides a connection whose details are described by the named network definition. Depending on the virtual network's fo rward mo d e configuration, the network may be totally isolated (no <fo rward > element given), NAT'ing to an explicit network device or to the default route (fo rward mo d e= ' nat' ), routed with no NAT (fo rward mo d e= ' ro ute' /), or connected directly to one of the host physical machine's network interfaces (via macvtap) or bridge devices (fo rward mo d e= ' bri d g e| pri vate| vepa| passthro ug h' /)
For networks with a forward mode of bridge, private, vepa, and passthrough, it is assumed that the host physical machine has any necessary DNS and DHCP services already setup outside the scope of libvirt. In the case of isolated, nat, and routed networks, DHCP and DNS are provided on the virtual network by libvirt, and the IP range can be determined by examining the virtual network config with vi rsh net-d umpxml [netwo rkname]. There is one virtual network called 'default' setup out of the box which does NAT'ing to the default route and has an IP range of
192.168.122.0/255.255.255.0. Each guest virtual machine will have an associated tun device created with a name of vnetN, which can also be overridden with the <targ et> element (refer to
Section 21.16.9.11, “ Overriding the target element” ).
When the source of an interface is a network, a portgroup can be specified along with the name of the network; one network may have multiple portgroups defined, with each portgroup containing slightly different configuration information for different classes of network connections. Also, similar to
<d i rect> network connections (described below), a connection of type netwo rk may specify a
<vi rtual po rt> element, with configuration data to be forwarded to a vepa (802.1Qbg) or
802.1Qbh compliant switch, or to an Open vSwitch virtual switch.
Since the actual type of switch may vary depending on the configuration in the <netwo rk> on the host physical machine, it is acceptable to omit the virtualport type attribute, and specify attributes from multiple different virtualport types (and also to leave out certain attributes); at domain startup time, a complete <vi rtual po rt> element will be constructed by merging together the type and attributes defined in the network and the portgroup referenced by the interface. The newlyconstructed virtualport is a combination of both. The attributes from lower virtualport can't make changes on the ones defined in higher virtualport. Interfaces take the highest priority, portgroup is lowest priority.
For example, in order to work properly with both an 802.1Qbh switch and an Open vSwitch switch, you may choose to specify no type, but both an pro fi l ei d (in case the switch is 802.1Qbh) and an i nterfacei d (in case the switch is Open vSwitch) (you may also omit the other attributes, such as manag eri d , typei d , or pro fi l ei d , to be filled in from the network's vi rtual po rt). If you want to limit a guest virtual machine to connecting only to certain types of switches, you can specify
373
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
the virtualport type, but still omit some/all of the parameters - in this case if the host physical machine's network has a different type of virtualport, connection of the interface will fail. The virtual network parameters are defined using management tools that modify the following part of the domain
XML:
...
<devices>
<interface type= 'network' >
<source network= 'default' />
</interface>
...
<interface type= 'network' >
<source network= 'default' portgroup= 'engineering' />
<target dev= 'vnet7' />
<mac address= "00:11:22:33:44:55" />
<virtualport>
<parameters instanceid= '09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f' />
</virtualport>
</interface>
</devices>
...
Fig u re 21.37. Devices - n et wo rk in t erf aces- virt u al n et wo rks
2 1 .1 6 .9 .2 . Bridge t o LAN
Note that this is the recommended configuration setting for general guest virtual machine connectivity on host physical machines with static wired networking configurations.
Bridge to LAN provides a bridge from the guest virtual machine directly onto the LAN. This assumes there is a bridge device on the host physical machine which has one or more of the host physical machines physical NICs enslaved. The guest virtual machine will have an associated tun device created with a name of <vnetN>, which can also be overridden with the <targ et> element (refer to
The IP range / network configuration is whatever is used on the LAN. This provides the guest virtual machine full incoming and outgoing net access just like a physical machine.
On Linux systems, the bridge device is normally a standard Linux host physical machine bridge. On host physical machines that support Open vSwitch, it is also possible to connect to an open vSwitch bridge device by adding a vi rtual po rt type= ' o penvswi tch' / to the interface definition. The
Open vSwitch type virtualport accepts two parameters in its parameters element - an i nterfacei d which is a standard uuid used to uniquely identify this particular interface to Open vSwitch (if you do no specify one, a random i nterfacei d will be generated for you when you first define the interface), and an optional pro fi l ei d which is sent to Open vSwitch as the interfaces <po rt-
pro fi l e>. To set the bridge to LAN settings, use a management tool that will configure the following part of the domain XML:
374
Chapt er 2 1 . Manipulat ing t he domain xml
...
<devices>
...
<interface type= 'bridge' >
<source bridge= 'br0' />
</interface>
<interface type= 'bridge' >
<source bridge= 'br1' />
<target dev= 'vnet7' />
<mac address= "00:11:22:33:44:55" />
</interface>
<interface type= 'bridge' >
<source bridge= 'ovsbr' />
<virtualport type= 'openvswitch' >
<parameters profileid= 'menial' interfaceid= '09b11c53-8b5c-4eeb-
8f00-d84eaa0aaa4f' />
</virtualport>
</interface>
...
</devices>
Fig u re 21.38. Devices - n et wo rk in t erf aces- b rid g e t o LAN
2 1 .1 6 .9 .3. Se t t ing a po rt m asque rading range
In cases where you want to set the port masquerading range, the port can be set as follows:
<forward mode= 'nat' >
<address start= '192.0.2.1' end= '192.0.2.10' />
</forward> ...
Fig u re 21.39 . Po rt Masq u erad in g Ran g e
These values should be set using the i ptabl es commands as shown in Section 19.2, “ Network
2 1 .1 6 .9 .4 . Use rspace SLIRP st ack
Setting the userspace SLIRP stack parameters provides a virtual LAN with NAT to the outside world.
The virtual network has DHCP and DNS services and will give the guest virtual machine an IP addresses starting from 10.0.2.15. The default router will be 10.0.2.2 and the DNS server will be
10.0.2.3. This networking is the only option for unprivileged users who need their guest virtual machines to have outgoing access.
The userspace SLIP stack parameters are defined in the following part of the domain XML::
375
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
...
<devices>
<interface type= 'user' />
...
<interface type= 'user' >
<mac address= "00:11:22:33:44:55" />
</interface>
</devices>
...
Fig u re 21.4 0. Devices - n et wo rk in t erf aces- Usersp ace SLIRP st ack
2 1 .1 6 .9 .5 . Ge ne ric Et he rne t co nne ct io n
Provides a means for the administrator to execute an arbitrary script to connect the guest virtual machine's network to the LAN. The guest virtual machine will have a tun device created with a name of vnetN, which can also be overridden with the targ et element. After creating the tun device a shell script will be run which is expected to do whatever host physical machine network integration is required. By default this script is called /etc/q emu-i fup but can be overridden (refer to
Section 21.16.9.11, “ Overriding the target element” ).
The generic Ethernet connection parameters are defined in the following part of the domain XML:
...
<devices>
<interface type= 'ethernet' />
...
<interface type= 'ethernet' >
<target dev= 'vnet7' />
<script path= '/etc/qemu-ifup-mynet' />
</interface>
</devices>
...
Fig u re 21.4 1. Devices - n et wo rk in t erf aces- g en eric Et h ern et co n n ect io n
2 1 .1 6 .9 .6 . Dire ct at t achm e nt t o physical int e rface s
Manipulating the direct attachment to physical interfaces provides direct attachment of the guest virtual machine's NIC to the given taht the physial interface of the host physical machine is specified.
This setup requires the Linux macvtap driver to be available. One of the modes vepa ( 'Virtual
Ethernet Port Aggregator'), bri d g e or pri vate can be chosen for the operation mode of the macvtap device, vepa being the default mode.
Manipulating direct attachment to physical interfaces involves setting the following parameters in the following part of the domain XML.
376
Chapt er 2 1 . Manipulat ing t he domain xml
...
<devices>
...
<interface type= 'direct' >
<source dev= 'eth0' mode= 'vepa' />
</interface>
</devices>
...
Fig u re 21.4 2. Devices - n et wo rk in t erf aces- d irect at t ach men t t o p h ysical in t erf aces
T ab le 21.17. Direct at t ach men t t o p h ysical in t erf ace elemen t s
Elemen t vepa bri d g e pri vate passthro ug h
Descrip t io n
All of the guest virtual machines' packets are sent to the external bridge. Packets whose destination is a guest virtual machine on the same host physical machine as where the packet originates from are sent back to the host physical machine by the VEPA capable bridge
(today's bridges are typically not VEPA capable).
Packets whose destination is on the same host physical machine as where they originate from are directly delivered to the target macvtap device. Both origin and destination devices need to be in bridge mode for direct delivery. If either one of them is in vepa mode, a VEPA capable bridge is required.
All packets are sent to the external bridge and will only be delivered to a target VM on the same host physical machine if they are sent through an external router or gateway and that device sends them back to the host physical machine.
This procedure is followed if either the source or destination device is in private mode.
This feature attaches a virtual function of a
SRIOV capable NIC directly to a guest virtual machine without losing the migration capability.
All packets are sent to the VF/IF of the configured network device. Depending on the capabilities of the device additional prerequisites or limitations may apply; for example, this requires kernel 2.6.38 or newer.
The network access of direct attached virtual machines can be managed by the hardware switch to which the physical interface of the host physical machine machine is connected to.
The interface can have additional parameters as shown below, if the switch is conforming to the IEEE
802.1Qbg standard. The parameters of the virtualport element are documented in more detail in the
IEEE 802.1Qbg standard. The values are network specific and should be provided by the network
377
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
administrator. In 802.1Qbg terms, the Virtual Station Interface (VSI) represents the virtual interface of a virtual machine.
Note that IEEE 802.1Qbg requires a non-zero value for the VLAN ID.
T ab le 21.18. Direct at t ach men t t o p h ysical in t erf ace ad d it io n al elemen t s
Elemen t manag eri d typei d typei d versi o n i nstancei d pro fi l ei d
Descrip t io n
The VSI Manager ID identifies the database containing the VSI type and instance definitions.
This is an integer value and the value 0 is reserved.
The VSI Type ID identifies a VSI type characterizing the network access. VSI types are typically managed by network administrator.
This is an integer value.
The VSI Type Version allows multiple versions of a VSI Type. This is an integer value.
The VSI Instance ID Identifier is generated when a VSI instance (i.e. a virtual interface of a virtual machine) is created. This is a globally unique identifier.
The profile ID contains the name of the port profile that is to be applied onto this interface.
This name is resolved by the port profile database into the network parameters from the port profile, and those network parameters will be applied to this interface.
Additional parameters in the domain XML include:
...
<devices>
...
<interface type= 'direct' >
<source dev= 'eth0.2' mode= 'vepa' />
<virtualport type= "802.1Qbg" >
<parameters managerid= "11" typeid= "1193047" typeidversion= "2" instanceid= "09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f" />
</virtualport>
</interface>
</devices>
...
Fig u re 21.4 3. Devices - n et wo rk in t erf aces- d irect at t ach men t t o p h ysical in t erf aces ad d it io n al p aramet ers
The interface can have additional parameters as shown below if the switch is conforming to the IEEE
378
Chapt er 2 1 . Manipulat ing t he domain xml
802.1Qbh standard. The values are network specific and should be provided by the network administrator.
Additional parameters in the domain XML include:
...
<devices>
...
<interface type= 'direct' >
<source dev= 'eth0' mode= 'private' />
<virtualport type= '802.1Qbh' >
<parameters profileid= 'finance' />
</virtualport>
</interface>
</devices>
...
Fig u re 21.4 4 . Devices - n et wo rk in t erf aces- d irect at t ach men t t o p h ysical in t erf aces mo re ad d it io n al p aramet ers
The pro fi l ei d attribute, contains the name of the port profile that is to be applied to this interface.
This name is resolved by the port profile database into the network parameters from the port profile, and those network parameters will be applied to this interface.
2 1 .1 6 .9 .7 . PCI passt hro ugh
A PCI network device (specified by the so urce element) is directly assigned to the guest virtual machine using generic device passthrough, after first optionally setting the device's MAC address to the configured value, and associating the device with an 802.1Qbh capable switch using an optionally specified vi rtual po rt element (see the examples of virtualport given above for type='direct' network devices). Note that - due to limitations in standard single-port PCI ethernet card driver design - only SR-IOV (Single Root I/O Virtualization) virtual function (VF) devices can be assigned in this manner; to assign a standard single-port PCI or PCIe ethernet card to a guest virtual machine, use the traditional ho std ev device definition
Note that this "intelligent passthrough" of network devices is very similar to the functionality of a standard ho std ev device, the difference being that this method allows specifying a MAC address and vi rtual po rt for the passed-through device. If these capabilities are not required, if you have a standard single-port PCI, PCIe, or USB network card that doesn't support SR-IOV (and hence would anyway lose the configured MAC address during reset after being assigned to the guest virtual machine domain), or if you are using a version of libvirt older than 0.9.11, you should use standard
ho std ev to assign the device to the guest virtual machine instead of i nterface
type= ' ho std ev' /.
...
<devices>
<interface type= 'hostdev' >
379
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<driver name= 'vfio' />
<source>
<address type= 'pci' domain= '0x0000' bus= '0x00' slot= '0x07' function= '0x0' />
</source>
<mac address= '52:54:00:6d:90:02' >
<virtualport type= '802.1Qbh' >
<parameters profileid= 'finance' />
</virtualport>
</interface>
</devices>
...
Fig u re 21.4 5. Devices - n et wo rk in t erf aces- PCI p asst h ro u g h
2 1 .1 6 .9 .8 . Mult icast t unne l
A multicast group may be used to represent a virtual network. Any guest virtual machine whose network devices are within the same multicast group will talk to each other, even if they reside across miltiple physical host physical machines. This mode may be used as an unprivileged user. There is no default DNS or DHCP support and no outgoing network access. To provide outgoing network access, one of the guest virtual machines should have a second NIC which is connected to one of the first 4 network types in order to provide appropriate routing. The multicast protocol is compatible with protocols used by user mo d e linux guest virtual machines as well. Note that the source address used must be from the multicast address block. A multicast tunnel is created by manipulating the i nterface type using a management tool and setting/changing it to mcast, and providing a mac and source address. The result is shown in changes made to the domain XML:
...
<devices>
<interface type= 'mcast' >
<mac address= '52:54:00:6d:90:01' >
<source address= '230.0.0.1' port= '5558' />
</interface>
</devices>
...
Fig u re 21.4 6 . Devices - n et wo rk in t erf aces- mu lt icast t u n n el
2 1 .1 6 .9 .9 . T CP t unne l
Creating a TCP client/server architecture is another way to provide a virtual network wher one guest virtual machine provides the server end of the network and all other guest virtual machines are configured as clients. All network traffic between the guest virtual machines is routed via the guest virtual machine that is configured as the server. This model is also available for use to unprivileged users. There is no default DNS or DHCP support and no outgoing network access. To provide outgoing network access, one of the guest virtual machines should have a second NIC which is
380
Chapt er 2 1 . Manipulat ing t he domain xml
connected to one of the first 4 network types thereby providing the appropriate routing. A TCP tunnel is created by manipulating the i nterface type using a management tool and setting/changing it to server or cl i ent, and providing a mac and source address. The result is shown in changes made to the domain XML:
...
<devices>
<interface type= 'server' >
<mac address= '52:54:00:22:c9:42' >
<source address= '192.168.0.1' port= '5558' />
</interface>
...
<interface type= 'client' >
<mac address= '52:54:00:8b:c9:51' >
<source address= '192.168.0.1' port= '5558' />
</interface>
</devices>
...
Fig u re 21.4 7. Devices - n et wo rk in t erf aces- T CP t u n n el
2 1 .1 6 .9 .1 0 . Se t t ing NIC drive r-spe cific o pt io ns
Some NICs may have tunable driver-specific options. These options are set as attributes of the
d ri ver sub-element of the interface definition. These options are set by using management tools to configuring the following sections of the domain XML:
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet1' />
<model type= 'virtio' />
<driver name= 'vhost' txmode= 'iothread' ioeventfd= 'on' event_idx= 'off' />
</interface>
</devices>
...
Fig u re 21.4 8. Devices - n et wo rk in t erf aces- set t in g NIC d river- sp ecif ic o p t io n s
Currently the following attributes are available for the "virtio" NIC driver:
T ab le 21.19 . virt io NIC d river elemen t s
381
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er name txmo d e i o eventfd event_i d x
Descrip t io n
The optional name attribute forces which type of backend driver to use. The value can be either
q emu (a user-space backend) or vho st (a kernel backend, which requires the vhost module to be provided by the kernel); an attempt to require the vhost driver without kernel support will be rejected. The default setting is vho st if the vhost driver present, but will silently fall back to q emu if not.
Specifies how to handle transmission of packets when the transmit buffer is full. The value can be either i o thread or ti mer. If set to i o thread , packet tx is all done in an iothread in the bottom half of the driver (this option translates into adding "tx= bh" to the q emu commandline device virtio-net-pci option). If set to ti mer, tx work is done in qemu, and if there is more tx data than can be sent at the present time, a timer is set before qemu moves on to do other things; when the timer fires, another attempt is made to send more data. In general you should leave this option alone, unless you are very certain you that changing it is an absolute necessity.
Allows users to set domain I/O asynchronous handling for interface device. The default is left to the discretion of the hypervisor. Accepted values are o n and o ff . Enabling this option allows qemu to execute a guest virtual machine while a separate thread handles I/O. Typically guest virtual machines experiencing high system CPU utilization during I/O will benefit from this. On the other hand, overloading the physical host physical machine may also increase guest virtual machine I/O latency.
Therefore, you should leave this option alone, unless you are very certain you that changing it is an absolute necessity.
The event_idx attribute controls some aspects of device event processing. The value can be either o n or o ff. Choosing o n, reduces the number of interrupts and exits for the guest virtual machine. The default is o n. In case there is a situation where this behavior is suboptimal, this attribute provides a way to force the feature off. You should leave this option alone, unless you are very certain you that changing it is an absolute necessity.
2 1 .1 6 .9 .1 1 . Ove rriding t he t arge t e le m e nt
To override the target element, use a management tool to make the following changes to the domain
XML:
382
Chapt er 2 1 . Manipulat ing t he domain xml
...
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet1' />
</interface>
</devices>
...
Fig u re 21.4 9 . Devices - n et wo rk in t erf aces- o verrid in g t h e t arg et elemen t
If no target is specified, certain hypervisors will automatically generate a name for the created tun device. This name can be manually specified, however the name must not start with either 'vnet' or
'vif', which are prefixes reserved by libvirt and certain hypervisors. Manually specified targets using these prefixes will be ignored.
2 1 .1 6 .9 .1 2 . Spe cifying bo o t o rde r
To specify the boot order, use a management tool to make the following changes to the domain XML:
...
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet1' />
<boot order= '1' />
</interface>
</devices>
...
Fig u re 21.50. Sp ecif yin g b o o t o rd er
For hypervisors which support it, you can set a specific NIC to be used for the network boot. The order of attributes determine the order in which devices will be tried during boot sequence. Note that the per-device boot elements cannot be used together with general boot elements in BIOS bootloader section.
2 1 .1 6 .9 .1 3. Int e rface ROM BIOS co nfigurat io n
To specify the ROM BIOS configuration settings, use a management tool to make the following changes to the domain XML:
...
383
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet1' />
<rom bar= 'on' file= '/etc/fake/boot.bin' />
</interface>
</devices>
...
Fig u re 21.51. In t erf ace RO M BIO S co n f ig u rat io n
For hypervisors which support it, you can change how a PCI Network device's ROM is presented to the guest virtual machine. The bar attribute can be set to o n or o ff, and determines whether or not the device's ROM will be visible in the guest virtual machine's memory map. (In PCI documentation, the "rombar" setting controls the presence of the Base Address Register for the ROM). If no rom bar is specified, the qemu default will be used (older versions of qemu used a default of o ff, while newer qemus have a default of o n). The optional fi l e attribute is used to point to a binary file to be presented to the guest virtual machine as the device's ROM BIOS. This can be useful to provide an alternative boot ROM for a network device.
2 1 .1 6 .9 .1 4 . Qualit y o f se rvice
This section of the domain XML provides setting quality of service. Incoming and outgoing traffic can be shaped independently. The band wi d th element can have at most one inbound and at most one outbound child elements. Leaving any of these children element out results in no QoS being applied on that traffic direction. Therefore, when you want to shape only domain's incoming traffic, use inbound only, and vice versa.
Each of these elements has one mandatory attribute averag e (or fl o o r as described below).
averag e specifies average bit rate on the interface being shaped. Then there are two optional attributes: peak, which specifies maximum rate at which interface can send data, and burst, which specifies the amount of bytes that can be burst at peak speed. Accepted values for attributes are integer numbers.
The units for averag e and peak attributes are kilobytes per second, whereas burst is only set in kilobytes. In addtion, inbound traffic can optionally have a fl o o r attribute. This guarantees minimal throughput for shaped interfaces. Using the fl o o r requires that all traffic goes through one point where QoS decisions can take place. As such it may only be used in cases where the i nterface
type= ' netwo rk' / with a fo rward type of ro ute, nat, or no forward at all). It should be noted that within a virtual network, all connected interfaces are required to have at least the inbound QoS set
(averag e at least) but the floor attribute doesn't require specifying averag e. However, peak and
burst attributes still require averag e. At the present time, ingress qdiscs may not have any classes, and therefore fl o o r may only be applied only on inbound and not outbound traffic.
To specify the QoS configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet0' />
384
Chapt er 2 1 . Manipulat ing t he domain xml
<bandwidth>
<inbound average= '1000' peak= '5000' floor= '200' burst= '1024' />
<outbound average= '128' peak= '256' burst= '256' />
</bandwidth>
</interface>
<devices>
...
Fig u re 21.52. Q u alit y o f service
2 1 .1 6 .9 .1 5 . Se t t ing VLAN t ag (o n suppo rt e d ne t wo rk t ype s o nly)
To specify the VLAN tag configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<interface type= 'bridge' >
<vlan>
<tag id= '42' />
</vlan>
<source bridge= 'ovsbr0' />
<virtualport type= 'openvswitch' >
<parameters interfaceid= '09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f' />
</virtualport>
</interface>
<devices>
...
Fig u re 21.53. Set t in g VLAN t ag ( o n su p p o rt ed n et wo rk t yp es o n ly)
If (and only if) the network connection used by the guest virtual machine supports vlan tagging transparent to the guest virtual machine, an optional vl an element can specify one or more vlan tags to apply to the guest virtual machine's network traffic (openvswitch and type= ' ho std ev' SR-
IOV interfaces do support transparent vlan tagging of guest virtual machine traffic; everything else, including standard Linux bridges and libvirt's own virtual networks, do not support it. 802.1Qbh (vnlink) and 802.1Qbg (VEPA) switches provide their own way (outside of libvirt) to tag guest virtual machine traffic onto specific vlans.) To allow for specification of multiple tags (in the case of vlan trunking), a subelement, tag , specifies which vlan tag to use (for example: tag i d = ' 4 2' /. If an interface has more than one vl an element defined, it is assumed that the user wants to do VLAN trunking using all the specified tags. In the case that vlan trunking with a single tag is desired, the optional attribute trunk= ' yes' can be added to the toplevel vlan element.
2 1 .1 6 .9 .1 6 . Mo difying virt ual link st at e
This element provides means of setting state of the virtual network link. Possible values for attribute
state are up and d o wn. If d o wn is specified as the value, the interface behaves as if it had the network cable disconnected. Default behavior if this element is unspecified is to have the link state
up.
385
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
To specify the virtual link state configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<interface type= 'network' >
<source network= 'default' />
<target dev= 'vnet0' />
<link state= 'down' />
</interface>
<devices>
...
Fig u re 21.54 . Mo d if yin g virt u al lin k st at e
21.16.10. Input devices
Input devices allow interaction with the graphical framebuffer in the guest virtual machine virtual machine. When enabling the framebuffer, an input device is automatically provided. It may be possible to add additional devices explicitly, for example, to provide a graphics tablet for absolute cursor movement.
To specify the input devices configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<input type= 'mouse' bus= 'usb' />
</devices>
...
Fig u re 21.55. In p u t d evices
The <i nput> element has one mandatory attribute: type which can be set to: mo use or tabl et. The latter provides absolute cursor movement, while the former uses relative movement. The optional bus attribute can be used to refine the exact device type and can be set to: xen (paravirtualized), ps2, and usb.
The input element has an optional sub-element <ad d ress>, which can tie the device to a particular
PCI slot, as documented above.
21.16.11. Hub devices
A hub is a device that expands a single port into several so that there are more ports available to connect devices to a host physical machine system.
386
Chapt er 2 1 . Manipulat ing t he domain xml
To specify the hub devices configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<hub type= 'usb' />
</devices>
...
Fig u re 21.56 . Hu b d evices
The hub element has one mandatory attribute, the type whose value can only be usb. The hub element has an optional sub-element ad d ress with type= ' usb' which can tie the device to a particular controller.
21.16.12. Graphical framebuffers
A graphics device allows for graphical interaction with the guest virtual machine OS. A guest virtual machine will typically have either a framebuffer or a text console configured to allow interaction with the admin.
To specify the graphical framebuffer devices configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
<graphics type= 'sdl' display= ':0.0' />
<graphics type= 'vnc' port= '5904' >
<listen type= 'address' address= '192.0.2.1' />
</graphics>
<graphics type= 'rdp' autoport= 'yes' multiUser= 'yes' />
<graphics type= 'desktop' fullscreen= 'yes' />
<graphics type= 'spice' >
<listen type= 'network' network= 'rednet' />
</graphics>
</devices>
...
Fig u re 21.57. G rap h ical f rameb u f f ers
The g raphi cs element has a mandatory type attribute which takes the value sd l , vnc, rd p or
d eskto p as explained below:
T ab le 21.20. G rap h ical f rameb u f f er elemen t s
387
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er sd l vnc spi ce
Descrip t io n
This displays a window on the host physical machine desktop, it can take 3 optional arguments: a d i spl ay attribute for the display to use, an xauth attribute for the authentication identifier, and an optional ful l screen attribute accepting values yes or no
Starts a VNC server. The po rt attribute specifies the TCP port number (with -1 as legacy syntax indicating that it should be auto-allocated). The
auto po rt attribute is the new preferred syntax for indicating autoallocation of the TCP port to use. The l i sten attribute is an IP address for the server to listen on. The passwd attribute provides a VNC password in clear text. The
keymap attribute specifies the keymap to use. It is possible to set a limit on the validity of the password be giving an ti mestamp
passwd Val i d T o = ' 20 10 -0 4 -
0 9 T 15: 51: 0 0 ' assumed to be in UTC. The
co nnected attribute allows control of connected client during password changes.
VNC accepts keep value only and note that it may not be supported by all hypervisors. Rather than using listen/port, QEMU supports a socket attribute for listening on a unix domain socket path.
Starts a SPICE server. The po rt attribute specifies the TCP port number (with -1 as legacy syntax indicating that it should be autoallocated), while tl sP o rt gives an alternative secure port number. The auto po rt attribute is the new preferred syntax for indicating autoallocation of both port numbers. The l i sten attribute is an IP address for the server to listen on. The passwd attribute provides a SPICE password in clear text. The keymap attribute specifies the keymap to use. It is possible to set a limit on the validity of the password be giving an ti mestamp passwd Val i d T o = ' 20 10 -
0 4 -0 9 T 15: 51: 0 0 ' assumed to be in UTC.
The co nnected attribute allows control of connected client during password changes.
SPICE accepts keep to keep client connected, disconnect to disconnect client and fail to fail changing password. Note it is not be supported by all hypervisors. The d efaul tMo d e attribute sets the default channel security policy, valid values are secure, i nsecure and the default
any (which is secure if possible, but falls back to i nsecure rather than erroring out if no secure path is available).
388
Chapt er 2 1 . Manipulat ing t he domain xml
When SPICE has both a normal and TLS secured TCP port configured, it may be desirable to restrict what channels can be run on each port. This is achieved by adding one or more channel elements inside the main g raphi cs element. Valid channel names include mai n, d i spl ay, i nputs, curso r,
pl ayback, reco rd ; smartcard ; and usbred i r.
To specify the SPICE configuration settings, use a mangement tool to make the following changes to the domain XML:
<graphics type= 'spice' port= '-1' tlsPort= '-1' autoport= 'yes' >
<channel name= 'main' mode= 'secure' />
<channel name= 'record' mode= 'insecure' />
<image compression= 'auto_glz' />
<streaming mode= 'filter' />
<clipboard copypaste= 'no' />
<mouse mode= 'client' />
</graphics>
Fig u re 21.58. SPICE co n f ig u rat io n
SPICE supports variable compression settings for audio, images and streaming. These settings are accessible via the compression attribute in all following elements: i mag e to set image compression
(accepts auto_glz, auto_lz, quic, glz, lz, off), jpeg for JPEG compression for images over wan
(accepts auto, never, always), zl i b for configuring wan image compression (accepts auto, never, always) and pl ayback for enabling audio stream compression (accepts on or off).
Streaming mode is set by the streami ng element, settings its mo d e attribute to one of fi l ter, al l or o ff.
In addition, Copy and paste functionality (via the SPICE agent) is set by the cl i pbo ard element. It is enabled by default, and can be disabled by setting the co pypaste property to no .
Mouse mode is set by the mo use element, setting its mo d e attribute to one of server or cl i ent. If no mode is specified, the qemu default will be used (cl i ent mode).
Additional elements include:
T ab le 21.21. Ad d it io n al g rap h ical f rameb u f f er elemen t s
Paramet er rd p
Descrip t io n
Starts a RDP server. The port attribute specifies the TCP port number (with -1 as legacy syntax indicating that it should be auto-allocated). The autoport attribute is the new preferred syntax for indicating autoallocation of the TCP port to use.
The replaceUser attribute is a boolean deciding whether multiple simultaneous connections to the VM are permitted. The multiUser whether the existing connection must be dropped and a new connection must be established by the VRDP server, when a new client connects in single connection mode.
389
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er d eskto p l i sten
Descrip t io n
This value is reserved for VirtualBox domains for the moment. It displays a window on the host physical machine desktop, similarly to "sdl", but uses the VirtualBox viewer. Just like "sdl", it accepts the optional attributes display and fullscreen.
Rather than putting the address information used to set up the listening socket for graphics types vnc and spice in the g raphi cs, the
l i sten attribute, a separate subelement of
g raphi cs, called l i sten can be specified (see the examples above). l i sten accepts the following attributes:
type - Set to either address or network. This tells whether this listen element is specifying the address to be used directly, or by naming a network (which will then be used to determine an appropriate address for listening).
ad d ress - this attribute will contain either an
IP address or hostname (which will be resolved to an IP address via a DNS query) to listen on. In the "live" XML of a running domain, this attribute will be set to the IP address used for listening, even if
type= ' netwo rk' .
network - if type= ' netwo rk' , the network attribute will contain the name of a network in libvirt's list of configured networks. The named network configuration will be examined to determine an appropriate listen address. For example, if the network has an
IPv4 address in its configuration (e.g. if it has a forward type of route, nat, or no forward type (isolated)), the first IPv4 address listed in the network's configuration will be used. If the network is describing a host physical machine bridge, the first IPv4 address associated with that bridge device will be used, and if the network is describing one of the 'direct' (macvtap) modes, the first IPv4 address of the first forward dev will be used.
21.16.13. Video devices
A video device.
To specify the video devices configuration settings, use a management tool to make the following changes to the domain XML:
390
Chapt er 2 1 . Manipulat ing t he domain xml
...
<devices>
<video>
<model type= 'vga' vram= '8192' heads= '1' >
<acceleration accel3d= 'yes' accel2d= 'yes' />
</model>
</video>
</devices>
...
Fig u re 21.59 . Vid eo d evices
The g raphi cs element has a mandatory type attribute which takes the value "sdl", "vnc", "rdp" or
"desktop" as explained below:
T ab le 21.22. G rap h ical f rameb u f f er elemen t s
Paramet er vi d eo mo d el
acceleration
ad d ress
Descrip t io n
The vi d eo element is the container for describing video devices. For backwards compatibility, if no video is set but there is a
g raphi cs element in domain xml, then libvirt will add a default vi d eo according to the guest virtual machine type. If "ram" or "vram" are not supplied a default value is used.
This has a mandatory type attribute which takes the value vg a, ci rrus, vmvg a, xen,
vbo x, or q xl depending on the hypervisor features available. You can also provide the amount of video memory in kibibytes (blocks of
1024 bytes) using vram and the number of figure with heads.
If acceleration is supported it should be enabled using the accel 3d and accel 2d attributes in the accel erati o n element.
The optional address sub-element can be used to tie the video device to a particular PCI slot.
21.16.14 . Consoles, serial, parallel, and channel devices
A character device provides a way to interact with the virtual machine. Paravirtualized consoles, serial ports, parallel ports and channels are all classed as character devices and so represented using the same syntax.
To specify the consols, channel and other devices configuration settings, use a management tool to make the following changes to the domain XML:
...
<devices>
391
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<parallel type= 'pty' >
<source path= '/dev/pts/2' />
<target port= '0' />
</parallel>
<serial type= 'pty' >
<source path= '/dev/pts/3' />
<target port= '0' />
</serial>
<console type= 'pty' >
<source path= '/dev/pts/4' />
<target port= '0' />
</console>
<channel type= 'unix' >
<source mode= 'bind' path= '/tmp/guestfwd' />
<target type= 'guestfwd' address= '10.0.2.1' port= '4600' />
</channel>
</devices>
...
Fig u re 21.6 0. Co n so les, serial, p arallel, an d ch an n el d evices
In each of these directives, the top-level element name (parallel, serial, console, channel) describes how the device is presented to the guest virtual machine. The guest virtual machine interface is configured by the target element. The interface presented to the host physical machine is given in the type attribute of the top-level element. The host physical machine interface is configured by the source element. The source element may contain an optional seclabel to override the way that labelling is done on the socket path. If this element is not present, the security label is inherited from the per-domain setting. Each character device element has an optional sub-element ad d ress which can tie the device to a particular controller or PCI slot.
21.16.15. Guest virt ual machine int erfaces
A character device presents itself to the guest virtual machine as one of the following types.
To set the parallel port, use a management tool to make the following change to the domain XML
...
<devices>
<parallel type= 'pty' >
<source path= '/dev/pts/2' />
<target port= '0' />
</parallel>
</devices>
...
Fig u re 21.6 1. G u est virt u al mach in e in t erf ace Parallel Po rt
<targ et> can have a po rt attribute, which specifies the port number. Ports are numbered starting from 0. There are usually 0, 1 or 2 parallel ports.
392
Chapt er 2 1 . Manipulat ing t he domain xml
To set the serial port use a management tool to make the following change to the domain XML:
...
<devices>
<serial type= 'pty' >
<source path= '/dev/pts/3' />
<target port= '0' />
</serial>
</devices>
...
Fig u re 21.6 2. G u est virt u al mach in e in t erf ace serial p o rt
<targ et> can have a po rt attribute, which specifies the port number. Ports are numbered starting from 0. There are usually 0, 1 or 2 serial ports. There is also an optional type attribute, which has two choices for its value, one is i sa-seri al , the other is usb-seri al . If type is missing, i sa-
seri al will be used by default. For usb-serial an optional sub-element <ad d ress> with
type= ' usb' can tie the device to a particular controller, documented above.
The <co nso l e> element is used to represent interactive consoles. Depending on the type of guest virtual machine in use, the consoles might be paravirtualized devices, or they might be a clone of a serial device, according to the following rules:
If no targ etT ype attribute is set, then the default device type is according to the hypervisor's rules. The default type will be added when re-querying the XML fed into libvirt. For fully virtualized guest virtual machines, the default device type will usually be a serial port.
If the targ etT ype attribute is seri al , and if no <seri al > element exists, the console element will be copied to the <seri al > element. If a <seri al > element does already exist, the console element will be ignored.
If the targ etT ype attribute is not seri al , it will be treated normally.
Only the first <co nso l e> element may use a targ etT ype of seri al . Secondary consoles must all be paravirtualized.
On s390, the console element may use a targetType of sclp or sclplm (line mode). SCLP is the native console type for s390. There's no controller associated to SCLP consoles.
In the example below, a virtio console device is exposed in the guest virtual machine as /dev/hvc[0-7]
(for more information, see http://fedoraproject.org/wiki/Features/VirtioSerial):
...
<devices>
<console type= 'pty' >
<source path= '/dev/pts/4' />
<target port= '0' />
</console>
393
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<!-- KVM virtio console -->
<console type= 'pty' >
<source path= '/dev/pts/5' />
<target type= 'virtio' port= '0' />
</console>
</devices>
...
...
<devices>
<!-- KVM s390 sclp console -->
<console type= 'pty' >
<source path= '/dev/pts/1' />
<target type= 'sclp' port= '0' />
</console>
</devices>
...
Fig u re 21.6 3. G u est virt u al mach in e in t erf ace - virt io co n so le d evice
If the console is presented as a serial port, the <targ et> element has the same attributes as for a serial port. There is usually only one console.
21.16.16. Channel
This represents a private communication channel between the host physical machine and the guest virtual machine and is manipulated by making changes to your guest virtual machine virtual machine using a management tool that results in changes made to the following section of the domain xml
...
<devices>
<channel type= 'unix' >
<source mode= 'bind' path= '/tmp/guestfwd' />
<target type= 'guestfwd' address= '10.0.2.1' port= '4600' />
</channel>
<!-- KVM virtio channel -->
<channel type= 'pty' >
<target type= 'virtio' name= 'arbitrary.virtio.serial.port.name' />
</channel>
<channel type= 'unix' >
<source mode= 'bind' path= '/var/lib/libvirt/qemu/f16x86_64.agent' />
<target type= 'virtio' name= 'org.qemu.guest_agent.0' />
</channel>
<channel type= 'spicevmc' >
<target type= 'virtio' name= 'com.redhat.spice.0' />
</channel>
</devices>
...
394
Chapt er 2 1 . Manipulat ing t he domain xml
Fig u re 21.6 4 . Ch an n el
This can be implemented in a variety of ways. The specific type of <channel > is given in the type attribute of the <targ et> element. Different channel types have different target attributes as follows:
g uestfwd - Dictates that TCP traffic sent by the guest virtual machine to a given IP address and port is forwarded to the channel device on the host physical machine. The targ et element must have address and port attributes.
vi rti o - Paravirtualized virtio channel. <channel > is exposed in the guest virtual machine under /d ev/vpo rt*, and if the optional element nameis specified, /d ev/vi rti o -po rts/$name
(for more info, please see http://fedoraproject.org/wiki/Features/VirtioSerial). The optional element
ad d ress can tie the channel to a particular type= ' vi rti o -seri al ' controller, documented above. With QEMU, if name is "org.qemu.guest_agent.0", then libvirt can interact with a guest virtual machine agent installed in the guest virtual machine, for actions such as guest virtual machine shutdown or file system quiescing.
spi cevmc - Paravirtualized SPICE channel. The domain must also have a SPICE server as a graphics device, at which point the host physical machine piggy-backs messages across the main channel. The targ et element must be present, with attribute type= ' vi rti o ' ; an optional attribute name controls how the guest virtual machine will have access to the channel, and defaults to name= ' co m. red hat. spi ce. 0 ' . The optional <ad d ress> element can tie the channel to a particular type= ' vi rti o -seri al ' controller.
21.16.17. Host physical machine int erface
A character device presents itself to the host physical machine as one of the following types:
T ab le 21.23. Ch aract er d evice elemen t s
Paramet er
Domain logfile
Descrip t io n
Disables all input on the character device, and sends output into the virtual machine's logfile
XML sn ip p et
<d evi ces>
<co nso l e type= ' std i o ' >
<targ et po rt= ' 1' />
</co nso l e>
</d evi ces>
395
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er
Device logfile
Descrip t io n
A file is opened and all data sent to the character device is written to the file.
XML sn ip p et
<d evi ces>
<seri al type= "fi l e">
<so urce path= "/var/l o g /vm/vm
-seri al . l o g "/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
Virtual console
Null device
Connects the character device to the graphical framebuffer in a virtual console. This is typically accessed via a special hotkey sequence such as
"ctrl+alt+3"
<d evi ces>
<seri al type= ' vc' >
<targ et po rt= "1"/>
</seri al >
</d evi ces>
Connects the character device to the void. No data is ever provided to the input. All data written is discarded.
<d evi ces>
<seri al type= ' nul l ' >
<targ et po rt= "1"/>
</seri al >
</d evi ces>
396
Chapt er 2 1 . Manipulat ing t he domain xml
Paramet er
Pseudo TTY
Descrip t io n
A Pseudo TTY is allocated using /d ev/ptmx. A suitable client such as vi rsh
co nso l e can connect to interact with the serial port locally.
XML sn ip p et
<d evi ces>
<seri al type= "pty">
<so urce path= "/d ev/pts/3"/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
NB Special case
Host physical machine device proxy
NB special case if <co nso l e
type= ' pty' >, then the TTY path is also duplicated as an attribute tty= ' /d ev/pts/3' on the top level <co nso l e> tag. This provides compat with existing syntax for <co nso l e> tags.
The character device is passed through to the underlying physical character device. The device types must match, eg the emulated serial port should only be connected to a host physical machine serial port don't connect a serial port to a parallel port.
<d evi ces>
<seri al type= "d ev">
<so urce path= "/d ev/ttyS0 "/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
397
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er
Named pipe
TCP client/server
Descrip t io n
The character device writes output to a named pipe. See pipe(7) MAN page for more info.
XML sn ip p et
<d evi ces>
<seri al type= "pi pe">
<so urce path= "/tmp/mypi pe"/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
The character device acts as a
TCP client connecting to a remote server.
<d evi ces>
<seri al type= "tcp">
<so urce mo d e= "co nnect" ho st= "0 . 0 . 0 . 0 " servi ce= "24 4 5"/>
<pro to co l type= "raw"/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
Or as a TCP server waiting for a client connection.
<d evi ces>
<seri al type= "tcp">
<so urce mo d e= "bi nd " ho st= "127. 0 . 0 . 1" servi ce= "24 4 5"/>
<pro to co l type= "raw"/>
<targ et po rt= "1"/>
398
Paramet er Descrip t io n
Chapt er 2 1 . Manipulat ing t he domain xml
XML sn ip p et
</seri al >
</d evi ces>
Alternatively you can use telnet instead of raw TCP. In addition, you can also use telnets
(secure telnet) and tls.
<d evi ces>
<seri al type= "tcp">
<so urce mo d e= "co nnect" ho st= "0 . 0 . 0 . 0 " servi ce= "24 4 5"/>
<pro to co l type= "tel net"/>
<targ et po rt= "1"/>
</seri al >
<seri al type= "tcp">
<so urce mo d e= "bi nd " ho st= "127. 0 . 0 . 1" servi ce= "24 4 5"/>
<pro to co l type= "tel net"/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
399
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Paramet er
UDP network console
UNIX domain socket client/server
Descrip t io n
The character device acts as a
UDP netconsole service, sending and receiving packets.
This is a lossy service.
The character device acts as a
UNIX domain socket server, accepting connections from local clients.
XML sn ip p et
<d evi ces>
<seri al type= "ud p">
<so urce mo d e= "bi nd " ho st= "0 . 0 . 0 . 0 " servi ce= "24 4 5"/>
<so urce mo d e= "co nnect" ho st= "0 . 0 . 0 . 0 " servi ce= "24 4 5"/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
<d evi ces>
<seri al type= "uni x">
<so urce mo d e= "bi nd " path= "/tmp/fo o "/>
<targ et po rt= "1"/>
</seri al >
</d evi ces>
21.17. Sound devices
A virtual sound card can be attached to the host physical machine via the sound element.
...
<devices>
<sound model= 'es1370' />
</devices>
...
Fig u re 21.6 5. Virt u al so u n d card
4 00
Chapt er 2 1 . Manipulat ing t he domain xml
The sound element has one mandatory attribute, mo d el , which specifies what real sound device is emulated. Valid values are specific to the underlying hypervisor, though typical choices are
' es1370 ' , ' sb16 ' , ' ac9 7' , and ' i ch6 ' . In addition, a sound element with ich6 model can have optional sub-elements co d ec to attach various audio codecs to the audio device. If not specified, a default codec will be attached to allow playback and recording. Valid values are ' d upl ex'
(advertises a line-in and a line-out) and ' mi cro ' (advertises a speaker and a microphone).
...
<devices>
<sound model= 'ich6' >
<codec type= 'micro' />
<sound/>
</devices>
...
Fig u re 21.6 6 . So u n d d evices
Each sound element has an optional sub-element <ad d ress> which can tie the device to a particular PCI slot, documented above.
21.18. Wat chdog device
A virtual hardware watchdog device can be added to the guest virtual machine via the <watchd o g > element. The watchdog device requires an additional driver and management daemon in the guest virtual machine. As merely enabling the watchdog in the libvirt configuration does not do anything useful on its own. Currently there is no support notification when the watchdog fires.
...
<devices>
<watchdog model= 'i6300esb' />
</devices>
...
...
<devices>
<watchdog model= 'i6300esb' action= 'poweroff' />
</devices>
< /domain>
Fig u re 21.6 7. Wat ch d o g d evice
The following attributes are declared in this XML:
mo d el - The required mo d el attribute specifies what real watchdog device is emulated. Valid values are specific to the underlying hypervisor.
4 01
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The mo d el attribute may take the following values:
i 6 30 0 esb — the recommended device, emulating a PCI Intel 6300ESB
i b70 0 — emulates an ISA iBase IB700
acti o n - The optional acti o n attribute describes what action to take when the watchdog expires. Valid values are specific to the underlying hypervisor. The acti o n attribute can have the following values:
reset — default setting, forcefully resets the guest virtual machine
shutd o wn — gracefully shuts down the guest virtual machine (not recommended)
po wero ff — forcefully powers off the guest virtual machine
pause — pauses the guest virtual machine
no ne — does nothing
d ump — automatically dumps the guest virtual machine.
Note that the 'shutdown' action requires that the guest virtual machine is responsive to ACPI signals.
In the sort of situations where the watchdog has expired, guest virtual machines are usually unable to respond to ACPI signals. Therefore using 'shutdown' is not recommended. In addition, the directory to save dump files can be configured by auto_dump_path in file /etc/libvirt/qemu.conf.
21.19. Memory balloon device
A virtual memory balloon device is added to all Xen and KVM/QEMU guest virtual machines. It will be seen as <membal l o o n> element. It will be automatically added when appropriate, so there is no need to explicitly add this element in the guest virtual machine XML unless a specific PCI slot needs to be assigned. Note that if the memballoon device needs to be explicitly disabled, mo d el = ' no ne' may be used.
The following example automatically added device with KVM
...
<devices>
<memballoon model= 'virtio' />
</devices>
...
Fig u re 21.6 8. Memo ry b allo o n d evice
Here is an example where the device is added manually with static PCI slot 2 requested
...
4 02
Chapt er 2 1 . Manipulat ing t he domain xml
<devices>
<memballoon model= 'virtio' >
<address type= 'pci' domain= '0x0000' bus= '0x00' slot= '0x02' function= '0x0' />
</memballoon>
</devices>
< /domain>
Fig u re 21.6 9 . Memo ry b allo o n d evice ad d ed man u ally
The required mo d el attribute specifies what type of balloon device is provided. Valid values are specific to the virtualization platform are: ' vi rti o ' which is the default setting with the KVM hypervisor or ' xen' which is the default setting with the Xen hypervisor.
21.20. T PM devices
The TPM device enables a QEMU guest virtual machine to have access to TPM functionality. The
TPM passthrough device type provides access to the host physical machine's TPM for one QEMU guest virtual machine. No other software may be is using the TPM device, typically /d ev/tpm0 , at the time the QEMU guest virtual machine is started. The following domain XML example shows the usage of the TPM passthrough device
...
<devices>
<tpm model= 'tpm-tis' >
<backend type= 'passthrough' >
<backend path= '/dev/tpm0' />
</backend>
</tpm>
</devices>
...
Fig u re 21.70. T PM d evices
The mo d el attribute specifies what device model QEMU provides to the guest virtual machine. If no model name is provided, tpm-tis will automatically be chosen. The <backend > element specifies the type of TPM device. The following types are supported: ' passthro ug h' — uses the host physical machine's TPM device and ' passthro ug h' . This backend type requires exclusive access to a TPM device on the host physical machine. An example for such a device is /d ev/tpm0 . The filename is specified as path attribute of the source element. If no file name is specified then /d ev/tpm0 is automatically used.
21.21. Securit y label
The <secl abel > element allows control over the operation of the security drivers. There are three basic modes of operation, ' d ynami c' where libvirt automatically generates a unique security label,
' stati c' where the application/administrator chooses the labels, or ' no ne' where confinement is disabled. With dynamic label generation, libvirt will always automatically relabel any resources
4 03
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
associated with the virtual machine. With static label assignment, by default, the administrator or application must ensure labels are set correctly on any resources, however, automatic relabeling can be enabled if desired.
If more than one security driver is used by libvirt, multiple seclabel tags can be used, one for each driver and the security driver referenced by each tag can be defined using the attribute mo d el Valid input XML configurations for the top-level security label are:
<seclabel type= 'dynamic' model= 'selinux' />
<seclabel type= 'dynamic' model= 'selinux' >
<baselabel> system_u:system_r:my_svirt_t:s0 </baselabel>
</seclabel>
<seclabel type= 'static' model= 'selinux' relabel= 'no' >
<label> system_u:system_r:svirt_t:s0:c392,c662 </label>
</seclabel>
<seclabel type= 'static' model= 'selinux' relabel= 'yes' >
<label> system_u:system_r:svirt_t:s0:c392,c662 </label>
</seclabel>
<seclabel type= 'none' />
Fig u re 21.71. Secu rit y lab el
If no ' type' attribute is provided in the input XML, then the security driver default setting will be used, which may be either ' no ne' or ' d ynami c' . If a <basel abel > is set but no ' type' is set, then the type is presumed to be ' d ynami c' . When viewing the XML for a running guest virtual machine with automatic resource relabeling active, an additional XML element, i mag el abel , will be included. This is an output-only element, so will be ignored in user supplied XML documents.
The following elements can be manipulated with the following values:
type - Either stati c, d ynami c or no ne to determine whether libvirt automatically generates a unique security label or not.
mo d el - A valid security model name, matching the currently activated security model
rel abel - Either yes or no . This must always be yes if dynamic label assignment is used. With static label assignment it will default to no .
<l abel > - If static labelling is used, this must specify the full security label to assign to the virtual domain. The format of the content depends on the security driver in use:
SELi nux: a SELinux context.
AppArmo r: an AppArmor profile.
D AC : owner and group separated by colon. They can be defined both as user/group names or uid/gid. The driver will first try to parse these values as names, but a leading plus sign can used to force the driver to parse them as uid or gid.
4 04
Chapt er 2 1 . Manipulat ing t he domain xml
<basel abel > - If dynamic labelling is used, this can optionally be used to specify the base security label. The format of the content depends on the security driver in use.
<i mag el abel > - This is an output only element, which shows the security label used on resources associated with the virtual domain. The format of the content depends on the security driver in use When relabeling is in effect, it is also possible to fine-tune the labeling done for specific source file names, by either disabling the labeling (useful if the file lives on NFS or other file system that lacks security labeling) or requesting an alternate label (useful when a management application creates a special label to allow sharing of some, but not all, resources between domains). When a seclabel element is attached to a specific path rather than the top-level domain assignment, only the attribute relabel or the sub-element label are supported.
21.22. Example domain XML configurat ion
QEMU emulated guest virtual machine on x86_64
<domain type= 'qemu' >
<name> QEmu-fedora-i686 </name>
<uuid> c7a5fdbd-cdaf-9455-926a-d65c16db1809 </uuid>
<memory> 219200 </memory>
<currentMemory> 219200 </currentMemory>
<vcpu> 2 </vcpu>
<os>
<type arch= 'i686' machine= 'pc' > hvm </type>
<boot dev= 'cdrom' />
</os>
<devices>
<emulator> /usr/bin/qemu-system-x86_64 </emulator>
<disk type= 'file' device= 'cdrom' >
<source file= '/home/user/boot.iso' />
<target dev= 'hdc' />
<readonly/>
</disk>
<disk type= 'file' device= 'disk' >
<source file= '/home/user/fedora.img' />
<target dev= 'hda' />
</disk>
<interface type= 'network' >
<source network= 'default' />
</interface>
<graphics type= 'vnc' port= '-1' />
</devices>
</domain>
Fig u re 21.72. Examp le d o main XML co n f ig
KVM hardware accelerated guest virtual machine on i686
4 05
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
<domain type= 'kvm' >
<name> demo2 </name>
<uuid> 4dea24b3-1d52-d8f3-2516-782e98a23fa0 </uuid>
<memory> 131072 </memory>
<vcpu> 1 </vcpu>
<os>
<type arch= "i686" > hvm </type>
</os>
<clock sync= "localtime" />
<devices>
<emulator> /usr/bin/qemu-kvm </emulator>
<disk type= 'file' device= 'disk' >
<source file= '/var/lib/libvirt/images/demo2.img' />
<target dev= 'hda' />
</disk>
<interface type= 'network' >
<source network= 'default' />
<mac address= '24:42:53:21:52:45' />
</interface>
<graphics type= 'vnc' port= '-1' keymap= 'de' />
</devices>
</domain>
Fig u re 21.73. Examp le d o main XML co n f ig
4 06
Chapt er 2 2 . T roubleshoot ing
Chapter 22. Troubleshooting
This chapter covers common problems and solutions for Red Hat Enterprise Linux 6 virtualization issues.
Read this chapter to develop an understanding of some of the common problems associated with virtualization technologies. Troubleshooting takes practice and experience which are difficult to learn from a book. It is recommended that you experiment and test virtualization on Red Hat
Enterprise Linux 6 to develop your troubleshooting skills.
If you cannot find the answer in this document there may be an answer online from the virtualization
community. Refer to Section B.1, “ Online resources”
for a list of Linux virtualization websites.
22.1. Debugging and t roubleshoot ing t ools
This section summarizes the System Administrator applications, the networking utilities, and debugging tools. You can employ these standard System administration tools and logs to assist with troubleshooting:
kvm_stat - refer to
trace-cmd
ftrace Refer to the Red Hat Enterprise Linux Developer Guide
vmstat i o stat l so f systemtap crash sysrq sysrq t sysrq w
These networking tools can assist with troubleshooting virtualization networking problems:
i fco nfi g tcpd ump
The tcpd ump command 'sniffs' network packets. tcpd ump is useful for finding network abnormalities and problems with network authentication. There is a graphical version of tcpd ump named wi reshark.
brctl
brctl is a networking tool that inspects and configures the Ethernet bridge configuration in the
Linux kernel. You must have root access before performing these example commands:
# brctl show bridge-name bridge-id STP enabled interfaces
4 07
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
----------------------------------------------------------------------
------virtbr0 8000.feffffff yes eth0
# brctl showmacs virtbr0 port-no mac-addr local? aging timer
1 fe:ff:ff:ff:ff: yes 0.00
2 fe:ff:ff:fe:ff: yes 0.00
# brctl showstp virtbr0 virtbr0 bridge-id 8000.fefffffffff
designated-root 8000.fefffffffff
root-port 0 path-cost 0 max-age 20.00 bridge-max-age 20.00
hello-time 2.00 bridge-hello-time 2.00
forward-delay 0.00 bridge-forward-delay 0.00
aging-time 300.01
hello-timer 1.43 tcn-timer 0.00
topology-change-timer 0.00 gc-timer 0.02
Listed below are some other useful commands for troubleshooting virtualization.
st race is a command which traces system calls and events received and used by another process.
vn cviewer: connect to a VNC server running on your server or a virtual machine. Install
vn cviwer using the yum i nstal l vnc command.
vn cserver: start a remote desktop on your server. Gives you the ability to run graphical user interfaces such as virt-manager via a remote session. Install vn cserver using the yum i nstal l
vnc-server command.
22.2. Creat ing virsh dump files
Executing a vi rsh d ump command sends a request to dump the core of a guest virtual machine to a file so errors in the virtual machine can be diagnosed. Running this command may require you to manually ensure proper permissions on file and path specified by the argument co refi l epath. The
vi rsh d ump command is similar to a coredump (or the crash utility). To create the vi rsh d ump file, run:
#vi rsh d ump <d o mai n> <co refi l epath> [--bypass-cache] { [--l i ve] | [--
crash] | [--reset] } [--verbo se] [--memo ry-o nl y]
While the domain (guest virtual machine domain name) and corefilepath (location of the newly created core dump file) are mandatory, the following arguments are optional:
--l i ve creates a dump file on a running machine and doesn't pause it.
--crash stops the guest virtual machine and generates the dump file. The main difference is that the guest virtual machine will not be listed as Stopped, with the reason as Crashed. Note that in
virt - man ag er the status will be listed as Paused.
--reset will reset the guest virtual machine following a successful dump. Note, these three switches are mutually exclusive.
--bypass-cache uses O_DIRECT to bypass the file system cache.
4 08
Chapt er 2 2 . T roubleshoot ing
--memo ry-o nl y the dump file will be saved as an elf file, and will only include domain’s memory and cpu common register value. This option is very useful if the domain uses host devices directly.
--verbo se displays the progress of the dump
The entire dump process may be monitored using vi rsh d o mjo bi nfo command and can be canceled by running vi rsh d o mjo babo rt.
22.3. kvm_st at
The kvm_stat command is a python script which retrieves runtime statistics from the kvm kernel module. The kvm_stat command can be used to diagnose guest behavior visible to kvm. In particular, performance related issues with guests. Currently, the reported statistics are for the entire system; the behavior of all running guests is reported. To run this script you need to install the qemu-
kvm-tools package. Refer to the Red Hat Enterprise Linux Virtualization Host Configuration and Guest
Installation Guide.
The kvm_stat command requires that the kvm kernel module is loaded and d ebug fs is mounted. If either of these features are not enabled, the command will output the required steps to enable
d ebug fs or the kvm module. For example:
# kvm_stat
Please mount debugfs ('mount -t debugfs debugfs /sys/kernel/debug') and ensure the kvm modules are loaded
Mount d ebug fs if required:
# mount -t debugfs debugfs /sys/kernel/debug
kvm_st at o u t p u t
The kvm_stat command outputs statistics for all guests and the host. The output is updated until the command is terminated (using C trl +c or the q key).
# kvm_stat kvm statistics efer_reload 94 0 exits 4003074 31272 fpu_reload 1313881 10796 halt_exits 14050 259 halt_wakeup 4496 203 host_state_reload 1638354 24893 hypercalls 0 0 insn_emulation 1093850 1909 insn_emulation_fail 0 0 invlpg 75569 0 io_exits 1596984 24509 irq_exits 21013 363 irq_injections 48039 1222 irq_window 24656 870 largepages 0 0 mmio_exits 11873 0
4 09
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
mmu_cache_miss 42565 8 mmu_flooded 14752 0 mmu_pde_zapped 58730 0 mmu_pte_updated 6 0 mmu_pte_write 138795 0 mmu_recycled 0 0 mmu_shadow_zapped 40358 0 mmu_unsync 793 0 nmi_injections 0 0 nmi_window 0 0 pf_fixed 697731 3150 pf_guest 279349 0 remote_tlb_flush 5 0 request_irq 0 0 signal_exits 1 0 tlb_flush 200190 0
Exp lan at io n o f variab les: ef er_relo ad
The number of Extended Feature Enable Register (EFER) reloads.
exit s
The count of all VMEXIT calls.
f p u _relo ad
The number of times a VMENT R Y reloaded the FPU state. The fpu_rel o ad is incremented when a guest is using the Floating Point Unit (FPU).
h alt _exit s
Number of guest exits due to hal t calls. This type of exit is usually seen when a guest is idle.
h alt _wakeu p
Number of wakeups from a hal t.
h o st _st at e_relo ad
Count of full reloads of the host state (currently tallies MSR setup and guest MSR reads).
h yp ercalls
Number of guest hypervisor service calls.
in sn _emu lat io n
Number of guest instructions emulated by the host.
in sn _emu lat io n _f ail
Number of failed i nsn_emul ati o n attempts.
io _exit s
Number of guest exits from I/O port accesses.
4 10
Chapt er 2 2 . T roubleshoot ing irq _exit s
Number of guest exits due to external interrupts.
irq _in ject io n s
Number of interrupts sent to guests.
irq _win d o w
Number of guest exits from an outstanding interrupt window.
larg ep ag es
Number of large pages currently in use.
mmio _exit s
Number of guest exits due to memory mapped I/O (MMIO) accesses.
mmu _cach e_miss
Number of KVM MMU shadow pages created.
mmu _f lo o d ed
Detection count of excessive write operations to an MMU page. This counts detected write operations not of individual write operations.
mmu _p d e_z ap p ed
Number of page directory entry (PDE) destruction operations.
mmu _p t e_u p d at ed
Number of page table entry (PTE) destruction operations.
mmu _p t e_writ e
Number of guest page table entry (PTE) write operations.
mmu _recycled
Number of shadow pages that can be reclaimed.
mmu _sh ad o w_z ap p ed
Number of invalidated shadow pages.
mmu _u n syn c
Number of non-synchronized pages which are not yet unlinked.
n mi_in ject io n s
Number of Non-maskable Interrupt (NMI) injections to the guest.
n mi_win d o w
Number of guest exits from (outstanding) Non-maskable Interrupt (NMI) windows.
p f _f ixed
Number of fixed (non-paging) page table entry (PTE) maps.
4 11
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Number of fixed (non-paging) page table entry (PTE) maps.
p f _g u est
Number of page faults injected into guests.
remo t e_t lb _f lu sh
Number of remote (sibling CPU) Translation Lookaside Buffer (TLB) flush requests.
req u est _irq
Number of guest interrupt window request exits.
sig n al_exit s
Number of guest exits due to pending signals from the host.
t lb _f lu sh
Number of tl b_fl ush operations performed by the hypervisor.
Note
The output information from the kvm_stat command is exported by the KVM hypervisor as pseudo files located in the /sys/kernel /d ebug /kvm/ directory.
22.4 . Guest virt ual machine fails t o shut down
Traditionally, executing a vi rsh shutd o wn command causes a power button ACPI event to be sent, thus copying the same action as when someone presses a power button on a physical machine.
Within every physical machine, it is up to the OS to handle this event. In the past operating systems would just silently shutdown. Today, the most usual action is to show a dialog asking what should be done. Some operating systems even ignore this event completely, especially when no users are logged in. When such operating systems are installed on a guest virtual machine, running vi rsh
shutd o wn just does not work (it is either ignored or a dialog is shown on a virtual display). However, if a qemu-guest-agent channel is added to a guest virtual machine and this agent is running inside the guest virtual machine's OS, the vi rsh shutd o wn command will ask the agent to shutdown the guest OS instead of sending the ACPI event. The agent will call for a shutdown from inside the guest virtual machine OS and everything works as expected.
Pro ced u re 22.1. Co n f ig u rin g t h e g u est ag en t ch an n el in a g u est virt u al mach in e
1. Stop the guest virtual machine.
2. Open the Domain XML for the guest virtual machine and add the following snippet:
4 12
<channel type= 'unix' >
<source mode= 'bind' />
<target type= 'virtio' name= 'org.qemu.guest_agent.0' />
</channel>
Chapt er 2 2 . T roubleshoot ing
Fig u re 22.1. Co n f ig u rin g t h e g u est ag en t ch an n el
3. Start the guest virtual machine, by running vi rsh start [d o mai n].
4. Install qemu-guest-agent on the guest virtual machine (yum i nstal l q emu-g uest-ag ent) and make it run automatically at every boot as a service (qemu-guest-agent.service). Refer to
Chapter 11, QEMU-img and QEMU guest agent
for more information.
22.5. T roubleshoot ing wit h serial consoles
Linux kernels can output information to serial ports. This is useful for debugging kernel panics and hardware issues with video devices or headless servers. The subsections in this section cover setting up serial console output for host physical machines using the KVM hypervisor.
This section covers how to enable serial console output for fully virtualized guests.
Fully virtualized guest serial console output can be viewed with the vi rsh co nso l e command.
Be aware fully virtualized guest serial consoles have some limitations. Present limitations include: output data may be dropped or scrambled.
The serial port is called ttyS0 on Linux or C O M1 on Windows.
You must configure the virtualized operating system to output information to the virtual serial port.
To output kernel information from a fully virtualized Linux guest into the domain, modify the
/bo o t/g rub/g rub. co nf file. Append the following to the kernel line: console=tty0
console=ttyS0,115200.
title Red Hat Enterprise Linux Server (2.6.32-36.x86-64)
root (hd0,0)
kernel /vmlinuz-2.6.32-36.x86-64 ro root=/dev/volgroup00/logvol00 \
console=tty0 console=ttyS0,115200
initrd /initrd-2.6.32-36.x86-64.img
Reboot the guest.
On the host, access the serial console with the following command:
# virsh console
You can also use vi rt-manag er to display the virtual text console. In the guest console window, select Seri al 1 in T ext C o nso l es from the Vi ew menu.
22.6. Virt ualizat ion log files
Each fully virtualized guest log is in the /var/l o g /l i bvi rt/q emu/ directory. Each guest log is named as GuestName.log and will be periodically compressed once a size limit is reached.
If you encounter any errors with the Virtual Machine Manager, you can review the generated data in the vi rt-manag er. l o g file that resides in the $HO ME/. vi rt-manag er directory.
22.7. Loop device errors
4 13
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
If file-based guest images are used you may have to increase the number of configured loop devices.
The default configuration allows up to eight active loop devices. If more than eight file-based guests or loop devices are needed the number of loop devices configured can be adjusted in the
/etc/mo d pro be. d /directory. Add the following line: options loop max_loop=64
This example uses 64 but you can specify another number to set the maximum loop value. You may also have to implement loop device backed guests on your system. To use a loop device backed guests for a full virtualized system, use the phy: d evi ce or fi l e: fi l e commands.
22.8. Live Migrat ion Errors
There may be cases where a live migration causes the memory contents to be re-transferred over and over again. This process causes the guest to be in a state where it is constantly writing to memory and therefore will slow down migration. If this should occur, and the guest is writing more than several tens of MBs per second, then live migration may fail to finish (converge). This issue is not scheduled to be resolved at the moment for Red Hat Enterprise Linux 6, and is scheduled to be fixed in Red Hat Enterprise Linux 7.
The current live-migration implementation has a default migration time configured to 30ms. This value determines the guest pause time at the end of the migration in order to transfer the leftovers.
Higher values increase the odds that live migration will converge
22.9. Enabling Int el VT -x and AMD-V virt ualizat ion hardware ext ensions in BIOS
This section describes how to identify hardware virtualization extensions and enable them in your
BIOS if they are disabled.
The Intel VT-x extensions can be disabled in the BIOS. Certain laptop vendors have disabled the
Intel VT-x extensions by default in their CPUs.
The virtualization extensions cannot be disabled in the BIOS for AMD-V.
Refer to the following section for instructions on enabling disabled virtualization extensions.
Verify the virtualization extensions are enabled in BIOS. The BIOS settings for Intel VT or AMD-V are usually in the Ch ip set or Pro cesso r menus. The menu names may vary from this guide, the virtualization extension settings may be found in Securi ty Setti ng s or other non standard menu names.
Pro ced u re 22.2. En ab lin g virt u aliz at io n ext en sio n s in BIO S
1. Reboot the computer and open the system's BIOS menu. This can usually be done by pressing the d el ete key, the F1 key or Al t and F4 keys depending on the system.
2. En ab lin g t h e virt u aliz at io n ext en sio n s in BIO S
4 14
Chapt er 2 2 . T roubleshoot ing
Note
Many of the steps below may vary depending on your motherboard, processor type, chipset and OEM. Refer to your system's accompanying documentation for the correct information on configuring your system.
a. Open the P ro cesso r submenu The processor settings menu may be hidden in the
C hi pset, Ad vanced C P U C o nfi g urati o n or No rthbri d g e.
b. Enable Intel Vi rtual i zati o n T echno l o g y (also known as Intel VT-x). AMD -V extensions cannot be disabled in the BIOS and should already be enabled. The virtualization extensions may be labeled Vi rtual i zati o n Extensi o ns,
Vand erpo o l or various other names depending on the OEM and system BIOS.
c. Enable Intel VT-d or AMD IOMMU, if the options are available. Intel VT-d and AMD
IOMMU are used for PCI device assignment.
d. Select Save & Exi t.
3. Reboot the machine.
4. When the machine has booted, run cat /pro c/cpui nfo | g rep -E "vmx| svm".
Specifying --color is optional, but useful if you want the search term highlighted. If the command outputs, the virtualization extensions are now enabled. If there is no output your system may not have the virtualization extensions or the correct BIOS setting enabled.
22.10. KVM net working performance
By default, KVM virtual machines are assigned a virtual Realtek 8139 (rtl8139) NIC (network interface controller). Whereas Red Hat Enterprise Linux guests are assigned a virtio NIC by default, Windows guests or the guest type is not specified.
The rtl8139 virtualized NIC works fine in most environments,but this device can suffer from performance degradation problems on some networks, such as a 10 Gigabit Ethernet.
To improve performance, you can switch to the para-virtualized network driver.
Note
Note that the virtualized Intel PRO/1000 (e10 0 0 ) driver is also supported as an emulated driver choice. To use the e10 0 0 driver, replace vi rti o in the procedure below with e10 0 0 .
For the best performance it is recommended to use the vi rti o driver.
Pro ced u re 22.3. Swit ch in g t o t h e virt io d river
1. Shutdown the guest operating system.
2. Edit the guest's configuration file with the vi rsh command (where GUEST is the guest's name):
# virsh edit GUEST
4 15
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The vi rsh ed i t command uses the $ED IT O R shell variable to determine which editor to use.
3. Find the network interface section of the configuration. This section resembles the snippet below:
<interface type='network'>
[output truncated]
<model type='rtl8139' />
</interface>
4. Change the type attribute of the model element from 'rtl8139' to 'virtio'. This will change the driver from the rtl8139 driver to the e1000 driver.
<interface type='network'>
[output truncated]
<model type='virtio' />
</interface>
5. Save the changes and exit the text editor
6. Restart the guest operating system.
Creat in g n ew g u est s u sin g o t h er n et wo rk d rivers
Alternatively, new guests can be created with a different network driver. This may be required if you are having difficulty installing guests over a network connection. This method requires you to have at least one guest already created (possibly installed from CD or DVD) to use as a template.
1. Create an XML template from an existing guest (in this example, named Guest1):
# virsh dumpxml Guest1 > /tmp/guest-template.xml
2. Copy and edit the XML file and update the unique fields: virtual machine name, UUID, disk image, MAC address, and any other unique parameters. Note that you can delete the UUID and MAC address lines and virsh will generate a UUID and MAC address.
# cp /tmp/guest-template.xml /tmp/new-guest.xml
# vi /tmp/new-guest.xml
Add the model line in the network interface section:
<interface type='network'>
[output truncated]
<model type='virtio' />
</interface>
3. Create the new virtual machine:
# virsh define /tmp/new-guest.xml
# virsh start new-guest
22.11. Workaround for creat ing ext ernal snapshot s wit h libvirt
4 16
Chapt er 2 2 . T roubleshoot ing
There are two classes of snapshots for QEMU guests. Internal snapshots are contained completely within a qcow2 file, and fully supported by libvirt, allowing for creating, deleting, and reverting of snapshots. This is the default setting used by libvirt when creating a snapshot, especially when no option is specified. Although this file type takes a bit longer than others in creating the the snapshot, it is required by libvirt to use qcow2 disks. Another drawback to this file type is that qcow2 disks are not subject to receive improvements from QEMU.
External snapshots, on the other hand work with any type of original disk image, can be taken with no guest downtime, and are able to receive active improvements from QEMU. In libvirt, they are created when using the --d i sk-o nl y option to snapsho t-create-as (or when specifying an explicit XML file to snapsho t-create that does the same). At the moment external snapshots are a one-way operation as libvirt can create them but cannot do anything further with them. A workaround is described here .
22.12. Missing charact ers on guest console wit h Japanese keyboard
On a Red Hat Enterprise Linux 6 host, connecting a Japanese keyboard locally to a machine may result in typed characters such as the underscore (the _ character) not being displayed correctly in guest consoles. This occurs because the required keymap is not set correctly by default.
With Red Hat Enterprise Linux 3 and Red Hat Enterprise Linux 6 guests, there is usually no error message produced when pressing the associated key. However, Red Hat Enterprise Linux 4 and Red
Hat Enterprise Linux 5 guests may display an error similar to the following: atkdb.c: Unknown key pressed (translated set 2, code 0x0 on isa0060/serio0). atkbd.c: Use 'setkeycodes 00 <keycode>' to make it known.
To fix this issue in vi rt-manag er, perform the following steps:
Open the affected guest in vi rt-manag er.
Click View → Det ails.
Select D i spl ay VNC in the list.
Change Au t o to ja in the Keymap pull-down menu.
Click the Appl y button.
Alternatively, to fix this issue using the vi rsh ed i t command on the target guest:
Run vi rsh ed i t <targ et g uest>
Add the following attribute to the <graphics> tag: keymap = ' ja' . For example:
<graphics type='vnc' port='-1' autoport='yes' keymap= ' ja' />
22.13. Known Windows XP guest issues
If you perform device-add quickly followed by device-del using a Windows XP guest, the guest does not eject the device and instead it displays the following error: "The device (device name) cannot be stopped because of an unknown error. Since the device is still being used, do not remove it". It should be noted that newer Windows OS version guests as well as all known Linux guests do not
4 17
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
experience this problem. To prevent this issue from happening, wait to delete a device that you just added.
22.14 . Verifying virt ualizat ion ext ensions
Use this section to determine whether your system has the hardware virtualization extensions.
Virtualization extensions (Intel VT-x or AMD-V) are required for full virtualization.
1. Run the following command to verify the CPU virtualization extensions are available:
$ grep -E 'svm|vmx' /proc/cpuinfo
2. Analyze the output.
The following output contains a vmx entry indicating an Intel processor with the Intel VT-x extension: flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush
dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl
vmx est tm2 cx16 xtpr lahf_lm
The following output contains an svm entry indicating an AMD processor with the AMD-V extensions: flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush
mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext
3dnow pni cx16
lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
If any output is received, the processor has the hardware virtualization extensions. However in some circumstances manufacturers disable the virtualization extensions in BIOS.
The "fl ag s: " output content may appear multiple times, once for each hyperthread, core or
CPU on the system.
The virtualization extensions may be disabled in the BIOS. If the extensions do not appear or
3. En su re KVM su b syst em is lo ad ed
As an additional check, verify that the kvm modules are loaded in the kernel:
# lsmod | grep kvm
If the output includes kvm_i ntel or kvm_amd then the kvm hardware virtualization modules are loaded and your system meets requirements.
4 18
Chapt er 2 2 . T roubleshoot ing
Note
If the libvirt package is installed, the vi rsh command can output a full list of virtualization system capabilities. Run vi rsh capabi l i ti es as root to receive the complete list.
4 19
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
The Virtual Host Metrics Daemon (vhostmd)
vh o st md (the Virtual Host Metrics Daemon) allows virtual machines to see limited information about the host they are running on. This daemon is only supplied with Red Hat Enterprise Linux for SAP.
In the host, a daemon (vh o st md ) runs which writes metrics periodically into a disk image. This disk image is exported read-only to guest virtual machines. Guest virtual machines can read the disk image to see metrics. Simple synchronization stops guest virtual machines from seeing out of date or corrupt metrics.
The system administrator chooses which metrics are available for use on a per guest virtual machine basis. In addition, the system administrator may block one or more guest virtual machines from having any access to metric configurations.
Customers who want to use vh o st md and vm- d u mp - met rics therefore need subscriptions for
"RHEL for SAP Business Applications" to be able to subscribe their RHEL systems running SAP to the "RHEL for SAP" channel on the Customer Portal or Red Hat Subscription Management to install the packages. The following kbase article in the customer portal describes the the setup of vhostmd on RHEL: https://access.redhat.com/knowledge/solutions/41566
4 20
Addit ional resources
Additional resources
To learn more about virtualization and Red Hat Enterprise Linux, refer to the following resources.
B.1. Online resources
http://www.libvirt.org/ is the official website for the l i bvi rt virtualization API.
http://virt-manager.et.redhat.com/ is the project website for the Virt u al Mach in e Man ag er (virtmanager), the graphical application for managing virtual machines.
Open Virtualization Center http://www.openvirtualization.com
Red Hat Documentation http://www.redhat.com/docs/
Virtualization technologies overview http://virt.kernelnewbies.org
Red Hat Emerging Technologies group http://et.redhat.com
B.2. Inst alled document at ion
man vi rsh and /usr/share/d o c/l i bvi rt-<version-number> — Contains sub commands and options for the vi rsh virtual machine management utility as well as comprehensive information about the l i bvi rt virtualization library API.
/usr/share/d o c/g no me-appl et-vm-<version-number> — Documentation for the GNOME graphical panel applet that monitors and manages locally-running virtual machines.
/usr/share/d o c/l i bvi rt-pytho n-<version-number> — Provides details on the Python bindings for the l i bvi rt library. The l i bvi rt-pytho n package allows python developers to create programs that interface with the l i bvi rt virtualization management library.
/usr/share/d o c/pytho n-vi rti nst-<version-number> — Provides documentation on the
vi rt-i nstal l command that helps in starting installations of Fedora and Red Hat Enterprise
Linux related distributions inside of virtual machines.
/usr/share/d o c/vi rt-manag er-<version-number> — Provides documentation on the
Virtual Machine Manager, which provides a graphical tool for administering virtual machines.
4 21
Red Hat Ent erprise Linux 6 Virt ualizat ion Administ rat ion G uide
Revision History
Revisio n 1- 39 4
Version for 6.6 GA release
T h u rs O ct 9 2014 Dayle Parker
Revisio n 1- 39 2 T h u rs O ct 9 2014 Dayle Parker
Clarified wording of step 3 for QE feedback in Procedure 13.7.1. Creating a GlusterFS storage pool using virsh for BZ #970435.
Fixed typos in virtio-rng section.
Minor wording change for BZ #1067282.
Revisio n 1- 388 T h u rs O ct 9 2014 Dayle Parker
Removed arguments unsupported by Red Hat Enterprise Linux KVM for virsh snapshot commands in
"Creating Snapshots" section for BZ #1020489.
Revisio n 1- 387 T h u rs O ct 9 2014
Fixed typos in BZ #970435.
Trimmed old revision history entries.
Dayle Parker
Revisio n 1- 385 Wed O ct 8 2014 Dayle Parker
Added note about --guest in 15.13.6. Configuring virtual CPU count, fixed typos for BZ #977085.
Index
F f eed b ack
- contact information for this manual,
H h elp
-
getting help, Do You Need Help?
V virt u aliz at io n
- additional resources
-
useful websites, Online resources
Virt u aliz at io n
- additional resources
- installed documentation,
4 22
Scarica
Link pubblico aggiornato
Il link pubblico alla tua chat è stato aggiornato. Puoi visualizzare e gestire i tuoi link di chat dopo la registrazione qui.
Link pubblico del documento
Pubblicità
