sg247039

sg247039

4

Chapter 4.

Virtualized resource management

򐂰

򐂰

򐂰

򐂰

This chapter provides detailed information about the new features and their capabilities that are available in IBM

^

p5 servers. It discusses the following topics:

4.1, “Micro-Partitioning technology” on page 84

4.2, “Advanced Virtualization” on page 97

4.3, “Introduction to Virtual I/O Server” on page 104

4.4, “Virtual I/O Server and virtualization configuration” on page 117

If you are a system administrator who has responsibility for configuration and management of POWER5-based servers with these advanced capabilities, it is imperative that you become familiar with the aspects that this chapter describes before you run the system with these features enabled.

© Copyright IBM Corp. 2003, 2004, 2005. All rights reserved.

83

4.1 Micro-Partitioning technology

Micro-Partitioning technology allows the resource definition of a partition to allocate fractions of processors to the partition. On POWER4 systems, all partitions are considered

dedicated.

The processors that are assigned to a partition can only be in whole multiples and only used by that partition. On

POWER5 systems, you can choose between

dedicated

processor partitions and

shared

processor partitions using Micro-Partitioning technology. You can have both dedicated and shared processor partitions running on the same system at the same time. At the time of the writing of this book, you can have only one shared processor pool per system.

Micro-Partitioning technology allows for increased overall use of system resources by applying automatically only the required amount of processor resources that each partition needs. Resources can also be defined as increments greater than a single processor.

The hypervisor continually adjusts the amount of processor capacity that is allocated to each shared processor partition and any excess capacity that is unallocated based on current partition profiles within a shared pool. Tuning parameters allow the administrator extensive control over the amount of processor resources that each partition can use.

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

This section discusses the following topics of Micro-Partitioning technology:

“Shared processor partitions” on page 84

“Processing units of capacity” on page 86

“Capped and uncapped mode” on page 88

“Virtual processors” on page 89

“Dedicated processors” on page 91

“Capped and uncapped processing units” on page 93

“Dynamic processor deallocation and sparing” on page 96

4.1.1 Shared processor partitions

The virtualization of processors enables the creation of a partitioning model which is fundamentally different from the POWER4 systems where whole processors are assigned to partitions and are owned by them. In the new model, physical processors are abstracted into virtual processors that are then assigned to partitions. However, the underlying physical processors are shared by these partitions.

Virtual processor abstraction is implemented in the hardware and microcode.

From an operating system perspective, a virtual processor is indistinguishable from a physical processor. The key benefit of implementing partitioning in the

84

Partitioning Implementations for IBM

E server

p5 Servers

hardware allows any operating system to run on POWER5 technology with little or no changes. Optionally, for optimal performance, the operating system can be enhanced to exploit shared processor pools more in-depth (for instance, by voluntarily relinquishing processor cycles to the hardware when they are not needed). AIX 5L Version 5.3 is the first version of AIX 5L that includes such enhancements.

Figure 4-1 Create Logical Partition Profile - shared processors

Micro-Partitioning technology allows for multiple partitions to share one physical processor. Partitions using Micro-Partitioning technology are referred to as shared processor partitions.

A partition may be defined with a processor capacity as small as 10 processor units. This represents 1/10 of a physical processor. Each processor can be shared by up to 10 shared processor partitions. The shared processor partitions are dispatched and time-sliced on the physical processors that are under control of the hypervisor.

Micro-Partitioning technology is supported across the entire POWER5 product

line from the entry-level to the high-end systems. Table 4-1 on page 86 shows

the maximum number of logical partitions and shared processor partitions of the different models.

Chapter 4. Virtualized resource management

85

Table 4-1 Micro-Partitioning technology overview on

Sserver

p5 systems

Sserver

p5 servers p5-510 p5-520 p5-550 p5-570

Processors

2 2 4 16

Dedicated processor partitions

2 2 4 16

Shared processor partitions

10 20 40 160

p5-590

32

32

254

p5-595

64

64

254

Note: The maximums listed in Table 4-1 are supported by the hardware.

However, the practical limits based on production workload demands might be significantly lower.

Shared processor partitions still need dedicated memory, but the partition I/O requirements can be supported through Virtual Ethernet and Virtual SCSI. Using all virtualization features, up to 254 shared processor partitions are supported in p5-590 and p5-595 systems.

The shared processor partitions are created and managed by the HMC. When you start creating a partition, you have to choose between a shared processor

partition and a dedicated processor partition (see Figure 4-1 on page 85).

When setting up a partition, you have to define the resources that belong to the partition, such as memory and IO resources. For shared processor partitions you have to configure the following additional options:

򐂰

򐂰

Minimum, desired, and maximum processing units of capacity.

The processing sharing mode to capped or uncapped. If the partition is uncapped, you must set its variable capacity weight also.

򐂰 Minimum, desired and maximum virtual processors.

4.1.2 Processing units of capacity

Processing capacity can be configured in fractions of 1/100 of a processor. The minimum amount or processing capacity which has to be assigned to a partition is 1/10 of a processor.

On the HMC, processing capacity is specified in terms of

processing units

. The minimum capacity of 1/10 of a processor which has to be assigned to a partition is specified as 0.1 processing units. To assign a processing capacity representing 75% of a processor, 0.75 processing units are specified on the

HMC.

86

Partitioning Implementations for IBM

E server

p5 Servers

On a system with two processors a maximum of 2.0 processing units can be assigned to partition. Processing units specified on the HMC are used to quantify the minimum, desired, and maximum amount of processing capacity for a

partition (see Figure 4-2).

Figure 4-2 Choose desired, minimum, and maximum processing units

After a partition is activated, processing capacity is usually referred to as

capacity entitlement or entitled capacity. Figure 4-3 on page 88 shows a graphic

representation of the definitions of processor capacity.

Chapter 4. Virtualized resource management

87

Minimum Requirement

0.1 Processing Units

0.5 Processing Units

0.4 Processing Units

Processing Capacity

1 Physical Processor

1.0 Processing Units

Figure 4-3 Processing units of capacity

4.1.3 Capped and uncapped mode

The next step in defining a shared processor partition is to define whether the

partition is running in a capped or uncapped mode (see Figure 4-4 on page 89).

Capped mode

Uncapped mode

The processing units never exceed the assigned processing capacity.

The processing capacity can be exceeded when the shared pool has available resources.

When a partition is running in an uncapped mode, you must specify the uncapped weight of that partition.

If multiple uncapped logical partitions require idle processing units, the managed system distributes idle processing units to the logical partitions in proportion to each logical partition's uncapped weight. The higher the uncapped weight of a logical partition, the more processing units the logical partition gets.

The uncapped weight must be a whole number from 0 to 255. The default uncapped weight for uncapped logical partitions is 128. A partition's share is computed by dividing its variable capacity weight by the sum of the variable capacity weights for all uncapped partitions. If you set the uncapped weight at 0, the managed system treats the logical partition as a capped logical partition. A logical partition with an uncapped weight of 0 cannot use more processing units than those that are committed to the logical partition.

88

Partitioning Implementations for IBM

E server

p5 Servers

Figure 4-4 Specify processing sharing mode and weight

4.1.4 Virtual processors

Virtual processors are the whole number of concurrent operations that the operating system can use. The processing power can be conceptualized as being spread equally across these virtual processors. Selecting the optimal number of virtual processors depends on the workload in the partition. Some partitions benefit from greater concurrence, where other partitions require greater power.

򐂰

򐂰

By default, the number of processing units that you specify is rounded up to the minimum number of virtual processors that are needed to satisfy the assigned number of processing units. The default settings maintains a balance of virtual processors to processor units. For example:

If you specify 0.50 processing units, one virtual processor are assigned.

If you specify 2.25 processing units, three virtual processors are assigned.

You also can use the advanced tab in your partitions profile to change the default

configuration and to assign more virtual processors (see Figure 4-5 on page 90).

Chapter 4. Virtualized resource management

89

At the time of the writing of this book, the maximum number of virtual processors per partition is 64.

A logical partition in the shared processing pool has at least as many virtual processors as its assigned processing capacity. By making the number of virtual processors too small, you limit the processing capacity of an uncapped partition.

If you have a partition with 0.50 processing units and 1 virtual processor, the partition cannot exceed 1.00 processing units, because it can only run one job at a time, which cannot exceed 1.00 processing units. However, if the same partition with 0.50 processing units was assigned two virtual processors and processing resources were available, the partition could use an additional 1.50 processing units.

Figure 4-5 Specify number of virtual processors

90

Partitioning Implementations for IBM

E server

p5 Servers

4.1.5 Dedicated processors

Dedicated processors are whole processors that are assigned to a single partition. If you choose to assign dedicated processors to a logical partition, you

must assign at least one processor to that partition (see Figure 4-6).

Figure 4-6 Create Logical Partition Profile, dedicated processors

You cannot mix shared processors and dedicated processors in one partition.

By default, a powered-off logical partition using dedicated processors has its processors available to the shared processing pool. When the processors are in the shared processing pool, an uncapped partition that needs more processing power can use the idle processing resources. However, when you power on the dedicated partition while the uncapped partition is using the processors, the activated partition regains all of its processing resources. If you want to allow this partition the ability to collect shared pool statistics, you can do so under the

hardware tab (see Figure 4-7 on page 92).

Chapter 4. Virtualized resource management

91

Figure 4-7 Allow shared processor pool statistics access to a partition

To set if a partition is to use resources from the shared processor pool, this is done in the profile properties under the processors tab.

92

Partitioning Implementations for IBM

E server

p5 Servers

Figure 4-8 Allow idle processors to be shared

Note: You cannot disable the Allow idle processor to be shared function when

you create a partition. You need to open the properties for the created partition and change it on the processor tab.

4.1.6 Capped and uncapped processing units

The hypervisor schedules shared processor partitions from a set of physical processors that is called the

shared processor pool

. By definition, these processors are not associated with dedicated partitions.

In shared partitions, there is no fixed relationship between virtual processors and physical processors. The hypervisor can use any physical processor in the shared processor pool when it schedules the virtual processor. By default, it attempts to use the same physical processor, but this cannot always be guaranteed. The hypervisor uses the concept of a home node for virtual processors, enabling it to select the best available physical processor from a memory affinity perspective for the virtual processor that is to be scheduled.

Chapter 4. Virtualized resource management

93

Affinity scheduling is designed to preserve the content of memory caches, so that the working data set of a job can be read or written in the shortest time period possible. Affinity is actively managed by the hypervisor, because each partition has a completely different context. Currently, there is one shared processor pool, so all virtual processors are implicitly associated with the same pool.

Figure 4-9 shows the relationship between two partitions using a shared

processor pool of a single physical processor. One partition has two virtual processors, and the other, a single one. The figure also shows how the capacity entitlement is evenly divided over the number of virtual processors.

When you set up a partition profile, you set up the desired, minimum, and maximum values that you want for the profile. When a partition is started, the system chooses the partition's entitled processor capacity from this specified capacity range. The value that is chosen represents a commitment of capacity that is reserved for the partition. This capacity cannot be used to start another shared partition. Otherwise, capacity could be overcommitted.

LPAR 1 Capacity Entitlement 50

Virtual

Processor 1

Virtual

Processor 2

25 25

LPAR 2 Capacity Entitlement 40

Virtual

Processor 1

40

0.5 Processing Units

0.4 Processing Units

1 Physical Processor

1.0 Processing Units

Figure 4-9 Distribution of capacity entitlement on virtual processors

When starting a partition, preference is given to the desired value, but these values cannot always be used, because there may not be enough unassigned capacity in the system. In that case, a different value is chosen, which must be greater than or equal to the minimum capacity attribute. Otherwise, the partition cannot be started. The entitled processor capacity is distributed to the partitions

94

Partitioning Implementations for IBM

E server

p5 Servers

in sequence the partitions are started. For example a shared pool has 2.0 processing units available.

Partitions 1, 2, and 3 are activated in sequence

1. Partition 1 activated

Min. = 1.0, max = 2.0, desired = 1.5

Allocated capacity entitlement: 1.5

2. Partition 2 activated

Min. = 1.0, max = 2.0, desired = 1.0

Partition 2 does not start because the minimum capacity is not met

3. Partition 3 activated

Min. = 0.1, max = 1.0, desired = 0.8

Allocated capacity entitlement: 0.5

The maximum value is only used as an upper limit for dynamic operations.

Figure 4-10 shows the usage of a capped partition of the shared processor pool.

Partitions using the capped mode are not able to assign more processing capacity from the shared processor pool than the capacity entitlement will allow.

Pool Idle Capacity Available

Maximum Processor Capacity

Processor

Capacity

Utilization

Entitled Processor Capacity

Minimum Processor Capacity

Utilized Capacity

Time

Figure 4-10 Shared capped processor partition utilization

Utilization

Chapter 4. Virtualized resource management

95

Figure 4-11 on page 96 shows the usage of the shared processor pool by an

uncapped partition. The uncapped partition is able to assign idle processing capacity if it needs more than the entitled capacity.

Pool Idle Capacity Available

Maximum Processor Capacity

Processor

Capacity

Utilization

Entitled Processor Capacity

Ceded Capacity

Minimum Processor Capacity

Utilized Capacity

Time

Figure 4-11 Shared uncapped processor partition utilization

4.1.7 Dynamic processor deallocation and sparing

Dynamic Processor Deallocation and Dynamic Processor Sparing is supported in the shared processor pool.

If a physical processor reaches a failure threshold and needs to be taken offline

(guarded out), the hypervisor analyzes the system environment to determine what action it should take to replace the processor resource. The options for handling this condition can be one of the following:

򐂰 If there is a Capacity on Demand processor available, the hypervisor switches the processor to the shared pool transparently, and no partition loss of capacity results.

򐂰 If there is at least 1.0 unallocated processor capacity available, it can be used to replace the capacity lost due to the failing processor.

If not enough unallocated resource exists, the hypervisor determines the capacity that each partition must lose to eliminate the 1.00 processor units from the shared pool. As soon as each partition varies off the processing capacity or

96

Partitioning Implementations for IBM

E server

p5 Servers

virtual processors, the failing processor is taken offline by the service processor and the hypervisor.

The amount of capacity that each shared partition is requested to vary off is proportional to the total amount of entitled capacity in the partition. This amount is based on the amount of capacity that can be varied off, which is controlled by the

min

capacity of the partition. The larger the difference between the current entitled capacity of the partition and the minimum entitled capacity, the more the partition will be asked to vary off.

Note: Dynamic Memory allocation or de-allocation is not supported by Linux

on

Sserver

p5 servers.

4.2 Advanced Virtualization

With the usage of virtual partitions in POWER5-based servers, the number of partitions that can be concurrently instantiated on a

Sserver

p5 server can be greater than the number of physical I/O slots in the Central Electronic Complex and its remote I/O drawers. For example, at the time of this publication,

Sserver p5 processor-based servers support 254 logical partitions, while p5-595 with remote I/O drawers can have up to 240 I/O slots.

Typically, a small operating system instance needs at least one slot for a network interface connector and one slot for a disk adapter (SCSI, Fibre Channel), while more robust configuration often consist of two redundant network interface connector adapters and two disk adapters.

To be able to connect enough I/O devices to each partition that is configured on a

Sserver

p5 server, IBM introduced virtual I/O technology for POWER5-based servers. Virtual I/O devices are an optional feature of a partition and can be used on POWER5 based systems in conjunction with AIX 5L Version 5.3 or Linux.

Virtual I/O devices are intended as a complement to physical I/O adapters (also known as dedicated or local I/O devices). A partition can have any combination of local and virtual I/O adapters.

򐂰

򐂰

The generic term

Virtual I/O

relates to five different concepts:

򐂰 Three adapters: Virtual SCSI, Virtual Ethernet, and Virtual Serial

A special AIX partition, called the Virtual I/O Server

A mechanism to link virtual network devices to real devices, called Shared

Ethernet Adapter (SEA).

Chapter 4. Virtualized resource management

97

4.2.1 Virtual LAN

Virtual LAN (VLAN) is a technology used for establishing virtual network segments on top of physical switch devices. If configured appropriately, a VLAN definition can straddle multiple switches. Typically, a VLAN is a broadcast domain that enables all nodes in the VLAN to communicate with each other without L3

routing or inter-VLAN bridging. In Figure 4-12, two VLANs (1 and 2) are defined

on three switches (A, B, and C). Although nodes C-1 and C-2 are physically connected to the same switch C, traffic between the two nodes can be blocked.

To enable communication between VLAN 1 and 2, L3 routing or inter-VLAN bridging should be established between them, typically by an L3 device.

Figure 4-12 Example of a VLAN

The use of VLAN provides increased LAN security and flexible network deployment over traditional network devices. VLANs provide additional security by allowing an administrator to block packets to a domain from a separate

98

Partitioning Implementations for IBM

E server

p5 Servers

domain on the same switch, therefore providing an additional control on what

LAN traffic is visible to specific Ethernet ports on the switch.

򐂰

򐂰

򐂰

򐂰

AIX virtual LAN support

Some of the various technologies for implementing VLANs include:

Port-based VLAN

Layer 2 VLAN

Policy-based VLAN

IEEE 802.1Q VLAN

VLAN support in AIX is based on the IEEE 802.1Q VLAN implementation. The

IEEE 802.1Q VLAN is achieved by adding a VLAN ID tag to an Ethernet frame, and the Ethernet switches restrict the frames to ports that are authorized to receive frames with that VLAN ID. Switches also restrict broadcasts to the logical network by ensuring that a broadcast packet is delivered to all ports which are configured to receive frames with the VLAN ID with which the broadcast frame was tagged.

A port on a VLAN capable switch has a default Port virtual LAN ID (PVID) that indicates the default VLAN to which the port belongs. The switch adds the PVID tag to untagged packets that are received by that port. In addition to a PVID, a port may belong to additional VLANs and have those VLAN IDs assigned to it that indicates the additional VLANs to which the port belongs.

A port only accepts untagged packets or packets with a VLAN ID (PVID or additional VIDs) tag of the VLANs to which the port belongs. A port configured in the untagged mode is only allowed to have a PVID and receives untagged packets or packets tagged with the PVID. The untagged port feature helps systems that do not understand VLAN tagging to communicate with other systems using standard Ethernet.

Each VLAN ID is associated with a separate Ethernet interface to the upper layers (for example IP) and creates unique logical Ethernet adapter instances per

VLAN (for example ent1 or ent2).

You can configure multiple VLAN logical devices on a single system. Each VLAN logical devices constitutes an additional Ethernet adapter instance. These logical devices can be used to configure the same Ethernet IP interfaces as are used with physical Ethernet adapters.

4.2.2 VLAN communication by example

This section discusses in more detail VLAN communication between partitions and with external networks and uses the sample configuration that is shown in

Chapter 4. Virtualized resource management

99

Figure 4-13 on page 100. The configuration uses four client partitions (Partition 1

through Partition 4) and one Virtual I/O Server. Each of the client partitions is defined with one Virtual Ethernet adapter. The Virtual I/O Server has an SEA which bridges traffic to the external network.

Figure 4-13 VLAN configuration

Interpartition communication

Partition 2 and Partition 4 use the PVID only, which means that:

򐂰 Only packets for the VLAN that is specified as PVID are received

򐂰 Packets sent are added a VLAN tag for the VLAN that is specified as PVID by the Virtual Ethernet adapter

򐂰

򐂰

In addition to the PVID, the Virtual Ethernet adapters in Partition 1 and Partition 3 are also configured for VLAN 10 using a specific network interface (en1) that was created with the

smitty vlan

command, which means that:

򐂰

Packets sent through network interfaces en1 are added a tag for VLAN 10 by the network interface in AIX

Only packets for VLAN 10 are received by the network interfaces en1

Packets sent through en0 are tagged automatically for the VLAN that is specified as PVID.

100

Partitioning Implementations for IBM

E server

p5 Servers

򐂰 Only packets for the VLAN that is specified as PVID are received by the network interfaces en0

Table 4-2 on page 101 lists which client partitions can communicate with each

other and the network interfaces that they can use.

Table 4-2 Interpartition of VLAN communication

VLAN Partition / Network interface

1

2

Partition 1 / en0

Partition 2 / en0

Partition 3 / en0

Partition 4 / en0

10 Partition 1 / en1

Partition 3 / en1

Communication with external networks

The SEA is configured with PVID 1 and VLAN 10. Untagged packets that are received by the SEA are tagged for VLAN 1. Handling of outgoing traffic depends on the VLAN tag of the outgoing packets. For example:

򐂰

򐂰

Packets tagged with the VLAN which match the PVID of the SEA are untagged before being sent out to the external network

Packets tagged with a VLAN

other

than that of the PVID of the SEA are sent out with the VLAN tag unmodified

In the example shown in Figure 4-13 on page 100, Partition 1 and Partition 2

have access to the external network through network interface en0 using VLAN

1. Because these packets are using the PVID, the SEA removes the VLAN tags before sending the packets to the external network.

Partition 1 and Partition 3 have access to the external network using network interface en1 and VLAN 10. Packets are sent out by the SEA with the VLAN tag.

Therefore, only VLAN-capable destination devices are able to receive the

packets. Table 4-3 lists this relationship.

Table 4-3 VLAN communication to external network

VLAN Partition / Network interface

1

10

Partition 1 / en0

Partition 2 / en0

Partition 1 / en1

Partition 3 / en1

Chapter 4. Virtualized resource management

101

Virtual Ethernet connections

Virtual Ethernet connections supported in POWER5 systems use VLAN technology to ensure that the partitions can access only data that is directed to them. The hypervisor provides a Virtual Ethernet switch function based on the

IEEE 802.1Q VLAN standard that allows partition communication within the same server. The connections are based on an implementation that is internal to

the hypervisor that moves data between partitions. Figure 4-14 is an example of

an inter-partition VLAN.

Figure 4-14 logical view of an inter-partition VLAN

Virtual Ethernet adapter concepts

Partitions that communicate through a Virtual Ethernet channel need to have an additional in-memory channel. This additional in-memory channel requires the creation of an in-memory channel between partitions on the HMC. The kernel creates a virtual device for each memory channel indicated by the firmware. The

AIX configuration manager creates the device special files. A unique media access control (MAC) address is also generated when the Virtual Ethernet device is created. You can assign a prefix value for the system so that the generated MAC addresses in a system consists of a common system prefix plus an algorithmically-generated unique part per adapter.

102

Partitioning Implementations for IBM

E server

p5 Servers

The Virtual Ethernet can also be used as a bootable device to allow such tasks as operating system installations to be performed using network installation management (NIM).

Performance considerations

The transmission speed of Virtual Ethernet adapters is in the range of 1 to 3

Gigabits per second, depending on the transmission (maximum transmission unit) size. A partition can support up to 256 Virtual Ethernet adapters with each

Virtual Ethernet capable to be associated with up to 21 VLANs (20 VID and 1

PVID).

The Virtual Ethernet connections generally take up more processor time than a local adapter to move a packet (DMA versus copy). For shared processor partitions, performance is gated by the partition definitions (for example, entitled capacity and number of processors). Small partitions communicating with each other experience more packet latency due to partition context switching. In general, high bandwidth applications

should not

be deployed in small shared processor partitions. For dedicated partitions, throughput

should be

comparable to a 1 Gigabit Ethernet for small packets providing much better performance than

1 Gigabit Ethernet for large packets. For large packets, the Virtual Ethernet communication is copy bandwidth limited.

Benefits of virtual Ethernet

Due to the number of partitions possible on many systems being greater than the number of I/O slots, Virtual Ethernet is a convenient and cost saving option to enable partitions within a single system to communicate with one another through a VLAN. The VLAN creates logical Ethernet connections between one or more partitions and is designed to help avoid a failed or malfunctioning operating system from being able to impact the communication between two functioning operating systems. The Virtual Ethernet connections may also be bridged to an external network to permit partitions without physical network adapters to communicate outside of the server.

Dynamic partitioning for virtual Ethernet devices

Virtual Ethernet resources can be assigned and removed dynamically. On the

HMC, Virtual Ethernet target and server adapters can be assigned and removed from a partition using dynamic logical partitioning. The mapping between physical and virtual resources on the Virtual I/O Server can also be done dynamically.

Limitations and considerations

The following are limitations that you should consider when implementing a

Virtual Ethernet:

Chapter 4. Virtualized resource management

103

򐂰

򐂰

򐂰

򐂰

򐂰

A maximum of up to 256 Virtual Ethernet adapters are permitted per partition.

Virtual Ethernet can be used in both shared and dedicated processor partitions, provided that the partition is running AIX 5L Version 5.3 or Linux with the 2.6 kernel or a kernel that supports virtualization.

A mixture of Virtual Ethernet connections, real network adapters, or both are permitted within a partition.

Virtual Ethernet can only connect partitions within a single system.

Virtual Ethernet requires a POWER5 system and an HMC to define the

Virtual Ethernet adapters.

Virtual Ethernet uses the system processors for all communication functions instead of off loading to processors on network adapter cards. As a result there is an increase in system processor load generated by the use of Virtual Ethernet.

4.3 Introduction to Virtual I/O Server

The Virtual I/O Server is an appliance that provides virtual storage and shared

Ethernet capability to client logical partitions on a POWER5 system. It allows a physical adapter with attached disks on the Virtual I/O Server partition to be shared by one or more partitions, enabling clients to consolidate and potentially minimize the number of physical adapters.

The Virtual I/O Server is the link between the virtual and the real world. It can be seen as an AIX-based appliance, and it is supported on POWER5 servers only.

The Virtual I/O Server runs in a special partition which cannot be used for execution of application code.

򐂰

򐂰

It mainly provides two functions:

Server component for Virtual SCSI devices (VSCI target)

Support of shared Ethernet adapters for Virtual Ethernet

The Virtual I/O Server is shipped as a mksysb image. Although the Virtual I/O

Server is based in AIX5L Version 5.3, it is not accessible as a standard partition.

Administrative access to the I/O Server partition is only possible as user padmin, not as id root. After login, user padmin gets a restricted shell, which is not escapable, called the I/O Server command line interface.

The operating system of the Virtual I/O Server is hidden to simplify transitions to later versions. Additionally, this product supports Linux and AIX 5L Version 5.3 client partitions. No specific operating system skill is required for administration of the Virtual I/O Server.

104

Partitioning Implementations for IBM

E server

p5 Servers

4.3.1 Shared Ethernet Adapter

You can use a Shared Ethernet Adapter (SEA) to connect a physical Ethernet to the Virtual Ethernet. An SEA also provides the possibility for several client partitions to share one physical adapter.

Connecting a virtual Ethernet to external networks

There are two ways you can connect the Virtual Ethernet that enables the communication between logical partitions on the same server to an external network.

Enabling routing capabilities

By enabling the AIX routing capabilities (ipforwarding network option), one partition with a physical Ethernet adapter connected to an external network can

act as router. Figure 4-15 shows a sample configuration. In this type of

configuration the partition that routes the traffic to the external work does not necessarily have to be the Virtual I/O Server as in the figure. It could be any partition with a connection to the outside world. The client partitions would have their default route set to the partition which routes traffic to the external network.

Figure 4-15 Connection to external network using AIX routing

Using Shared Ethernet Adapter

Using SEA, you can connect internal and external VLANs using one physical adapter. The SEA hosted in the Virtual I/O Server acts as a layer two switch between the internal and external network.

Chapter 4. Virtualized resource management

105

SEA is a new service that securely transports network traffic from a virtual

Ethernet to a real network adapter. The SEA service runs in the Virtual I/O

Server. It cannot be run in a general purpose AIX partition.

SEA requires the hypervisor component of POWER5 systems and therefore cannot be used on POWER4 systems. It also cannot be used with AIX 5L

Version 5.2, because the device drivers for virtual Ethernet are only available for

AIX 5L Version 5.3 and Linux. Thus, there is no way to connect an AIX 5L Version

5.2 system to an SEA.

򐂰

򐂰

򐂰

The SEA allows partitions to communicate outside the system without having to dedicate a physical I/O slot and a physical network adapter to a client partition.

The SEA has the following characteristics:

Virtual Ethernet MAC addresses that are visible to outside systems

Broadcast and multicast are supported

ARP and NDP can work across a shared Ethernet

To bridge network traffic between the Virtual Ethernet and external networks, the

Virtual I/O Server has to be configured with at least one physical Ethernet adapter. One SEA can be shared by multiple VLANs, and multiple subnets can

connect using a single adapter on the Virtual I/O Server. Figure 4-16 shows a

configuration example. An SEA can include up to 16 Virtual Ethernet adapters that share the physical access.

Figure 4-16 SEA configuration

A Virtual Ethernet adapter connected to the SEA must have the trunk flag set.

Once an Ethernet frame is sent from the Virtual Ethernet adapter on a client partition to the hypervisor, the hypervisor searches for the destination MAC

106

Partitioning Implementations for IBM

E server

p5 Servers

address within the VLAN. If no such MAC address exists within the VLAN, it forwards the frame to the trunk Virtual Ethernet adapter that is defined on the same VLAN. The trunk virtual Ethernet adapter enables a layer two bridge to a physical adapter.

The shared Ethernet directs packets based on the VLAN ID tags, based on observing the packets that originate from the virtual adapters. One of the virtual adapters in the SEA is designated as the default PVID adapter. Ethernet frames without any VLAN ID tags are directed to this adapter and assigned the default

PVID.

When the shared Ethernet receives IP (or IPv6) packets that are larger than the maximum transmission unit of the adapter that the packet is forwarded through, either IP fragmentation is performed and the fragments are forwarded, or an

ICMP packet too big message is returned to the source when the packet cannot be fragmented.

Theoretically, one adapter can act as the only contact with external networks for all client partitions. Because the SEA is dependant on Virtual I/O, it consumes processor time for all communications. An increased amount of processor load can be generated by the use of Virtual Ethernet and SEA.

There are several different ways to configure physical and virtual Ethernet adapters into SEAs to maximize throughput.

򐂰

Using Link Aggregation (EtherChannel), several physical network adapter can be aggregated.

򐂰 Using several SEAs provides more queues and more performance. An

example for this configuration is shown in Figure 4-17 on page 108.

Other aspects to take into consideration are availability and the possibility of connecting to different networks.

Chapter 4. Virtualized resource management

107

Figure 4-17 Multiple SEA configuration

Using Link Aggregation to external networks

Link aggregation is network port aggregation technology that allows several

Ethernet adapters to be aggregated together to form a single pseudo Ethernet device. You can use this technology to overcome the bandwidth limitation of a single network adapter and to avoid bottlenecks when sharing one network adapter among many client partitions.

For example, ent0 and ent1 can be aggregated to ent3. Interface en3 would then be configured with an IP address. The system considers these aggregated adapters as one adapter. Therefore, IP is configured as on any other Ethernet adapter. In addition, all adapters in the link aggregation are given the same MAC address, so that they are treated by remote systems as though they were one adapter. The main benefit of link aggregation is that they have the network bandwidth of all of their adapters in a single network presence. If an adapter fails, the packets are sent automatically on the next available adapter without disruption to existing user connections. The adapter is returned automatically to service on the link aggregation when it recovers.

You can use EtherChannel (EC) or IEEE 802.3ad Link Aggregation (LA) to aggregate network adapters. While EC is an AIX specific implementation of

adapter aggregation, LA follows the IEEE 802.3ad standard. Table 4-4 on page 109 shows the main differences between EC and LA.

108

Partitioning Implementations for IBM

E server

p5 Servers

Table 4-4 Main differences between EC and LA aggregation

EtherChannel IEEE 802.3ad link aggregation

Requires switch configuration

Supports different packet distribution modes

Little, if any, configuration of switch required to form aggregation. Some initial setup of the switch may be required.

Supports only standard distribution mode

Using LA, if the switch supports the Link Aggregation Control Protocol, then no special configuration of the switch ports is required. EC supports different packet distribution modes making it possible to influence the load balancing of the aggregated adapters. The remainder of this document uses the term

Link

Aggregation

because it is a more universally understood term.

Note: Only outgoing packets are subject to the following discussion. Incoming

packets are distributed by the Ethernet switch.

Standard distribution mode selects the adapter for the outgoing packets by algorithm. The adapter selection algorithm uses the last byte of the destination IP address (for TCP/IP traffic) or MAC address (for ARP and other non-IP traffic).

Therefore, all packets to a specific IP-address will always go through the same adapter. There are other adapter selection algorithms based on source, destination, or a combination of source and destination ports that are available.

EC provides one further distribution mode called

round robin.

This mode rotates through the adapters, giving each adapter one packet before repeating. The packets may be sent out in a slightly different order than they were given to the

EC. This method makes the best use of its bandwidth, but consider that it also introduces the potential for out-of-order packets at the receiving system. This risk is particularly high when there are few, long-lived, streaming TCP connections.

When there are many such connections between a host pair, packets from different connections could be intermingled, thereby decreasing the chance of packets for the same connection arriving out-of-order.

To avoid the loss of network connectivity by switch failure, EC and LA can provide a backup adapter. The backup adapter should be connected to a different switch than the adapter of the aggregation. So, in case of switch failure the traffic can be moved with no disruption of user connections to the backup adapter.

Figure 4-18 on page 110 shows the aggregation of three plus one adapters to a

single pseudo Ethernet device including a backup feature.

Chapter 4. Virtualized resource management

109

Figure 4-18 Link Aggregation (EtherChannel) pseudo device

Limitations and considerations

Consider the following limitations when implementing SEAs in the Virtual I/O

Server:

򐂰

Because SEA depends on virtual Ethernet which uses the system processors for all communications functions, an increased amount of system processor load can be generated by the use of Virtual Ethernet and SEA.

򐂰

򐂰

One of the virtual adapters in the SEA on the Virtual I/O Server must be defined as default adapter with a default PVID. This virtual adapter is designated as the PVID adapter and Ethernet frames without any VLAN ID tags are assigned the default PVID and directed to this adapter.

Up to 16 virtual Ethernet adapters with 21 VLANs (20 VID and 1 PVID) on each can be shared on a single physical network adapter. There is no limit on the number of partitions that can attach to a VLAN. So, the theoretical limit is very high. In practice, the amount of network traffic limits the number of clients that can be served through a single adapter.

For performance information please to Advanced POWER Virtualization on

IBM

^

p5 Servers Architecture and Performance Considerations,

SG24-5768.

110

Partitioning Implementations for IBM

E server

p5 Servers

4.3.2 Virtual SCSI

This section discusses Virtual SCSI technology.

Introduction to Virtual SCSI technology

Virtual SCSI requires POWER5 hardware with the Advanced POWER

Virtualization feature activated. It provides Virtual SCSI support for AIX 5L

Version 5.3 and Linux.

The driving forces behind Virtual SCSI are:

򐂰

The advanced technological capabilities of today’s hardware and operating systems such as POWER5 and IBM AIX 5L Version 5.3.

򐂰 The value proposition enabling on demand computing and server consolidation. Virtual SCSI also provides a more economic I/O model by using physical resources more efficiently through sharing.

At the time of the writing of this book, the virtualization features of the POWER5 platform support up to 254 partitions while the server hardware only provides up to 240 I/O slots per machine. Each partition typically requiring one I/O slot for disk attachment and another one for network attachment puts a constraint on the number of partitions. To overcome these physical limitations, I/O resources have to be shared. Virtual SCSI provides the means to do this for SCSI storage devices.

Furthermore Virtual SCSI allows attachment of previously unsupported storage solutions. As long as the Virtual I/O Server supports the attachment of a storage resource, any client partition can access this storage by using Virtual SCSI adapters. For example, you can run Linux in a logical partition of a POWER5 server to provide support for EMC storage devices on Linux.

A Linux client partition can access the EMC storage through a Virtual SCSI adapter. Requests from the virtual adapters are mapped to the physical resources in the Virtual I/O Server. Therefore, driver support for the physical resources is needed only in the Virtual I/O Server.

Chapter 4. Virtualized resource management

111

Note: This publication refers to different terms for the various components that

are involved with Virtual SCSI. Depending on the context, these terms can vary. With SCSI, usually the terms

initiator

and

target

are used. So, you might see terms such as the terms

virtual SCSI initiator

and

virtual SCSI target

. On the HMC,

virtual SCSI server adapter

and

virtual SCSI client adapter

are used. Basically, these terms refer to the same thing. When describing the client and server relationship between the partitions involved in Virtual SCSI, the terms

hosting partition

(meaning the Virtual I/O Server) and

hosted partition

(meaning the client partition) are used.

The terms

Virtual I/O Server partition

and

Virtual I/O Server

both refer to the

Virtual I/O Server

. The terms are used interchangeably in this section.

Partition access to virtual SCSI devices

Virtual SCSI is based on a client and server relationship. The Virtual I/O Server owns the physical resources and acts as a server or, in SCSI terms, a target device. The logical partitions access the virtual SCSI resources that are provided by the Virtual I/O Server as clients.

The virtual I/O adapters are configured using an HMC. The provisioning of virtual disk resources is provided by the Virtual I/O Server. Often the Virtual I/O Server is also referred to as

hosting

partition and the client partitions, as

hosted

partitions.

Physical disks owned by the Virtual I/O Server can either be exported and assigned to a client partition as whole or can be partitioned into several logical volumes. The logical volumes can then be assigned to different partitions.

Therefore, Virtual SCSI enables sharing of adapters as well as disk devices.

To make a physical or a logical volume available to a client partition it is assigned to a Virtual SCSI server adapter in the Virtual I/O Server.

The client partition accesses its assigned disks through a Virtual SCSI Client

Adapter. The Virtual SCSI Client Adapter sees standard SCSI devices and LUNs through this virtual adapter. The commands in the following example show how the disks appear on an AIX client partition:

# lsdev -Cc disk -s vscsi hdisk2 Available Virtual SCSI Disk Drive

# lscfg -vpl hdisk2 hdisk2 111.520.10DDEDC-V3-C5-T1-L810000000000 Virtual SCSI Disk Drive

112

Partitioning Implementations for IBM

E server

p5 Servers

In Figure 4-19, one physical disk is partitioned into two logical volumes inside the

Virtual I/O Server. Each of the two client partitions is assigned one logical volume which it accesses through a virtual I/O adapter (Virtual SCSI Client Adapter).

Inside the partition the disk is seen as a normal hdisk.

Figure 4-19 Virtual SCSI architecture overview

SCSI Remote Direct Memory Access

The SCSI family of standards provides many different transport protocols that define the rules for exchanging information between SCSI initiators and targets.

Virtual SCSI uses the SCSI RDMA Protocol, which defines the rules for exchanging SCSI information in an environment where the SCSI initiators and targets have the ability to directly transfer information between their respective address spaces.

SCSI requests and responses are sent using the Virtual SCSI adapters that communicate through the hypervisor. The actual data transfer, however, is done directly between a data buffer in the client partition and the physical adapter in the Virtual I/O Server by using the Logical Remote Direct Memory Access

(LRDMA) protocol.

Chapter 4. Virtualized resource management

113

Figure 4-20 shows how the data transfer using LRDMA appears.

Figure 4-20 Logical Remote Direct Memory Access

AIX device configuration for virtual SCSI

The virtual I/O adapters are connected to a virtual host bridge that AIX treats much like a PCI host bridge. It is represented in the object data model as a bus device whose parent is sysplanar0. The virtual I/O adapters are represented as adapter devices with the virtual host bridge as their parent.

On the Virtual I/O Server, each logical volume or physical volume that is exported to a client partition is represented by a virtual target device that is a child of a

Virtual SCSI server adapter. On the client partition, the exported disks are visible as normal hdisks. However, they are defined in subclass vscsi. They have a virtual SCSI client adapter as parent.

114

Partitioning Implementations for IBM

E server

p5 Servers

Figure 4-21 and Figure 4-22 on page 116 show the relationship of the devices

used by AIX for Virtual SCSI and their physical counterparts.

Figure 4-21 Virtual SCSI device relationship on Virtual I/O Server

Chapter 4. Virtualized resource management

115

Figure 4-22 Virtual SCSI device relationship on AIX client partition

Dynamic partitioning for virtual SCSI devices

Virtual SCSI resources can be assigned and removed dynamically. On the HMC,

Virtual SCSI target and server adapters can be assigned and removed from a partition using dynamic logical partitioning. The mapping between physical and virtual resources on the Virtual I/O Server can also be done dynamically.

4.3.3 Limitations and considerations

You should consider the following areas when implementing Virtual SCSI:

򐂰

򐂰

򐂰

At the time of the writing of this book, virtual SCSI supports Fibre Channel, parallel SCSI, and SCSI RAID devices. Other protocols such as SSA or tape and CD-ROM devices are not supported.

Virtual SCSI itself does not have any limitations in terms of the number of supported devices or adapters. At the time of writing, the Virtual I/O Server supports a maximum of 1024 virtual I/O slots on an IBM

Sserver

p5 server.

A maximum of 256 virtual I/O slots can be assigned to a single partition.

Every I/O slot needs some resources to be instantiated. Therefore, the size of the Virtual I/O Server puts a limit to the number of virtual adapters that can be configured. The SCSI protocol defines mandatory and optional commands.

While Virtual SCSI supports all the mandatory commands, not all optional commands are supported.

116

Partitioning Implementations for IBM

E server

p5 Servers

򐂰

򐂰

There are performance implications when using Virtual SCSI devices. It is important to understand that associated with hypervisor calls, Virtual SCSI uses additional processor cycles when processing I/O requests. Thus, when putting heavy I/O load on Virtual SCSI devices, you use more processor cycles. Provided that there is sufficient processing capacity available, the performance of Virtual SCSI should be comparable to dedicated I/O devices.

Suitable applications for Virtual SCSI include boot disks for the operating system or Web servers which will typically cache a lot of data. When designing a virtual I/O configuration, performance is an important aspect which should be given careful consideration.

For a more in depth discussion of performance issues see Advanced POWER

Virtualization on IBM

^

p5 Servers Architecture and Performance

Considerations, SG24-5768.

4.4 Virtual I/O Server and virtualization configuration

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

This section provides the following information about the configuration and operating environment of the Virtual I/O Server:

Command line interface

Hardware resources managed

Virtual I/O Server software installation

Basic configuration

Ethernet adapter sharing

Virtual SCSI disk

Defining the Virtual SCSI Server Adapter on the HMC

Defining the Virtual SCSI Client Adapter on the HMC

Creating the virtual target device on the Virtual I/O Server

Limitations and considerations

4.4.1 Using the command line interface

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

The Virtual I/O Server provides a restricted scriptable command line interface. All aspects of Virtual I/O server administration are accomplished through the command line interface, including:

Device management (physical, virtual, logical volume manager)

Network configuration

Software installation and update

Security

User management

Installation of OEM software

Maintenance tasks

Chapter 4. Virtualized resource management

117

For the initial log on to the Virtual I/O Server, use the user ID padmin, which is the prime administrator. When logging in, you are prompted for a new password, so there is no default password to remember.

Upon logging into the I/O server, you are placed in a restricted Korn shell. The restricted Korn shell works the same way as a regular Korn shell except users cannot:

򐂰

Change the current working directory

򐂰

򐂰

Set the value of the SHELL, ENV, or PATH variables

Specify the path name of the command that contains a redirect output of a command with a ‘>’, ‘>|’ , ‘<>’ or ‘>>’

As a result of these restrictions, you are not able to execute commands that are not accessible to your PATH. In addition, these restrictions prevent you from directly sending the output of the command to a file, requiring you to pipe the output to the

tee

command instead.

After you have logged on, you can type

help

to get an overview of the supported

commands as shown in Example 4-1.

Example 4-1 $help command for an overview

$ help

Install Commands Physical Volume Commands Security Commands

updateios lspv lsgcl

lssw migratepv cleargcl

ioslevel lsfailedlogin

remote_management Logical Volume Command

oem_setup_env lslv UserID Commands

oem_platform_level mklv mkuser

license extendlv rmuser

rmlvcopy lsuser

LAN Commands rmlv passwd

mktcpip mklvcopy chuser

hostname

cfglnagg

netstat Volume Group Commands Maintenance Commands

entstat lsvg chlang

cfgnamesrv mkvg diagmenu

traceroute chvg shutdown

ping extendvg fsck

optimizenet reducevg backupios

lsnetsvc mirrorios savevgstruct

unmirrorios restorevgstruct

Device Commands activatevg starttrace

mkvdev deactivatevg stoptrace

118

Partitioning Implementations for IBM

E server

p5 Servers

lsdev importvg cattracerpt

lsmap exportvg bootlist

chdev syncvg snap

rmdev startsysdump

cfgdev topas

mkpath mount

chpath unmount

lspath showmount

rmpath startnetsvc

errlog

stopnetsvc

To receive further help with these commands, you can use the

help

command as

shown in Example 4-2.

Example 4-2 $help command for a specific command

$ help errlog

Usage: errlog [-ls | -rm Days]

Displays or clears the error log.

-ls Displays information about errors in the error log file

in a detailed format.

-rm Deletes all entries from the error log older than the

number of days specified by the Days parameter.

򐂰

򐂰

The Virtual I/O Server command line interface supports two execution modes:

Traditional mode

Interactive mode

The traditional mode is for single command execution. In this mode, you execute one command at a time from the shell prompt. For example, to list all virtual devices, enter the following:

#ioscli lsdev -virtual

To reduce the amount of typing that is required in traditional shell level mode, an alias has been created for each sub-command. With the aliases set, you are not required to type the

ioscli

command. For example, to list all devices of type adapter, you can enter the following:

#lsdev -type adapter

Chapter 4. Virtualized resource management

119

In interactive mode the user is presented with the

ioscli

command prompt by executing the

ioscli

command without any sub-commands or arguments. From this point on,

ioscli

commands are executed one after the other without having to retype

ioscli

. For example, to enter interactive mode, enter:

#ioscli

Once in interactive mode, to list all virtual devices, enter:

#lsdev -virtual

External commands, such as

grep

or

sed

, cannot be executed from the interactive mode command prompt. You must first exit interactive mode by entering

quit

or

exit

.

4.4.2 Managing hardware resources

The optional Advanced POWER Virtualization feature that enables

Micro-Partitioning on a

Sserver

p5 server provides the Virtual I/O Server installation CD. A logical partition with enough resources to share to other partitions is required. Below is a list of minimum hardware requirements that must be available to create the Virtual I/O Server:

POWER5 server

HMC

The Virtual I/O capable machine.

To create the partition and assign resources.

Storage adapter

Physical disk

Ethernet adapter

Memory

The server partition needs at least one storage adapter.

If you want to share your disk with client partitions, you need a disk that is large enough to make sufficient-sized logical volumes.

If you want to route network traffic securely from a virtual

Ethernet to a real network adapter.

At least 128 MB of memory.

Virtual I/O Server Version 1.1 is designed for selected configurations that include specific models from IBM and other storage product vendors. Consult your IBM representative or Business Partner for the latest information and included configurations.

120

Partitioning Implementations for IBM

E server

p5 Servers

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

򐂰

Virtual devices that are exported to client partitions by the Virtual I/O Server must be attached through one of the following physical adapters:

PCI 4-Channel Ultra3 SCSI RAID Adapter (FC 2498)

PCI-X Dual Channel Ultra320 SCSI RAID Adapter (FC 5703)

Dual Channel SCSI RAID Enablement Card (FC 5709)

PCI-X Dual Channel Ultra320 SCSI Adapter (FC 5712)

2 Gigabit Fibre Channel PCI-X Adapter (FC 5716)

2 Gigabit Fibre Channel Adapter for 64-bit PCI Bus (FC 6228)

2 Gigabit Fibre Channel PCI-X Adapter (FC 6239)

We recommend that you plan carefully before you begin the configuration and installation of your I/O Server and client partitions. Depending on the type of workload and needs for an application, consider mixing virtual and physical devices. For example, if your application benefits from fast disk access, then plan a physical adapter that is dedicated to this partition.

Installation of the Virtual I/O Server partition is performed from a special mksysb

CD that is provided, at an additional charge, when you order the Advanced

POWER Virtualization feature. This CD contains dedicated software that is only for the Virtual I/O Server operations. So, the Virtual I/O server software is only supported in Virtual I/O Server partitions.

You can install the Virtual I/O Server from CD or using NIM on Linux from the

HMC. For more information about the installation of the Virtual I/O Server, refer to

4.4.3, “Installing Virtual I/O Server” on page 121.

򐂰

򐂰

򐂰

The Virtual I/O Server supports the following operating systems as Virtual I/O client:

IBM AIX 5L Version 5.3

SUSE LINUX Enterprise Server 9 for POWER

Red Hat Enterprise Linux AS for POWER Version 3

4.4.3 Installing Virtual I/O Server

This section describes how to install Virtual I/O Server to the partition called

I/O_Server_1. These instructions assume that you are familiar with a basic AIX installation.

Chapter 4. Virtualized resource management

121

Figure 4-23 shows an example of an HMC-based Virtual I/O server creation.

Figure 4-23 Hardware Management Console - create Virtual I/O server

To install Virtual I/O Server, follow these steps:

1. Activate the I/O_Server_1 partition by right-clicking the partition name and

selecting Activate, as shown in Figure 4-24.

Figure 4-24 Activate I/O_Server_1 partition

122

Partitioning Implementations for IBM

E server

p5 Servers

2. Select the default profile that you used to create this server. Select Open a

terminal window or console session, and click (Advanced...), as shown in

Figure 4-25.

Figure 4-25 Selecting the profile

3. Choose SMS boot mode as shown in Figure 4-26. Click OK in this window to

return to the previous window. When at the previous window, click OK to activate the partition and to launch a terminal window.

Figure 4-26 Choosing SMS boot mode

Chapter 4. Virtualized resource management

123

4. Figure 4-27 shows a pSeries SMS menu. Proceed with the installation as you

would for an AIX installation, and choose CD as the installation device.

Figure 4-27 SMS menu

124

Partitioning Implementations for IBM

E server

p5 Servers

5. When the installation procedure has finished, use padmin for the username at the login prompt. Choose a new password, and accept the license using the

license

command at the prompt ($), as shown in Figure 4-28. You can use

the

lspv

command to show the available disks.

Figure 4-28 Finished Virtual I/O Server installation

The I/O_Server_1 partition is now ready for the further configurations.

4.4.4 Basic configuration

The Virtual I/O Server provides the Virtual SCSI and SEA Virtual I/O to client partitions. You accomplish this configuration by assigning physical devices to the

Virtual I/O Server partition, and then by configuring virtual adapters on the clients to allow communication between the client and the Virtual I/O Server.

Chapter 4. Virtualized resource management

125

Using virtual I/O devices:

򐂰

Facilitates the sharing of physical resources between partitions on a

POWER5 system

򐂰

򐂰

Provides Virtual SCSI and SEA function to client partitions

Enables the creation of partitions without requiring additional physical I/O resources

򐂰

򐂰

Allows the creation of more partitions than I/O slots or physical devices with the ability for partitions to have dedicated I/O, virtual I/O, or both

Maximizes the utilization of physical resources on a POWER5 system

4.4.5 Ethernet adapter sharing

SEA enables the client partitions to communicate with other systems outside the

Central Electronic Complex without requiring physical Ethernet adapters in the partitions. This communication is accomplished by sharing the physical Ethernet adapters in the Virtual I/O Server partition.

VLANs that are bridged outside using a SEA, require a Virtual Ethernet adapter to have the trunk adapter setting on. This Virtual Ethernet adapter is assigned to the Virtual I/O Server partition using the HMC. The SEA setup commands are then run on the Virtual I/O Server to create associations between the physical and virtual adapters.

To configure a trunk Virtual Ethernet adapter on the HMC:

1. Right-click the partition profile of the Virtual I/O Server partition, and open the properties of the profile.

2. Choose the Virtual I/O tab. The HMC panel to create a virtual adapter

appears, as shown in Figure 4-29 on page 127.

126

Partitioning Implementations for IBM

E server

p5 Servers

Figure 4-29 Creating the trunk Virtual I/O Server

3. Make sure that the value in the Number of virtual adapters field is higher than the highest used Slot Number. To allow additional dynamically configured adapters, choose a number where you can add more virtual adapters later.

This value is similar to the maximum value of the processor and memory definition. To change it, you have to shutdown and reactivate the partition.

4. Select Ethernet in the Create Adapters panel, and then click (Create...). The

Virtual Ethernet Adapter Properties dialog box appears, as shown in

Figure 4-29.

The slot number that is used for this adapter identifies the virtual adapter within the logical partition. The combination of the slot number and the logical partition LAN ID uniquely identifies this slot within the managed system.

Chapter 4. Virtualized resource management

127

Figure 4-30 Virtual Ethernet Adapter Properties panel

Each Virtual Ethernet adapter has assigned a Port virtual LAN ID (PVID) or a virtual LAN ID (VID) number. Selecting the IEEE 802.1Q compatible adapter option allows you to configure additional virtual LAN IDs. This allows the Virtual

Ethernet adapter to be part of additional VLANs as specified in the IEEE 802.1Q network standard. Virtual Ethernet adapters can communicate with each other only if they are assigned to the same PVID or VID number.

Important: Trunk adapter must be selected on each Virtual Ethernet adapter

that will be mapped to create an SEA.

5. After you define a virtual adapter in a partition profile, you must shutdown and reactivate the partition to make the adapter available. If a network connection is already configured and your RMC connection is working correctly, you can add the Virtual Ethernet adapter dynamically to the partition.

128

Partitioning Implementations for IBM

E server

p5 Servers

6. Right-click the partition name, and choose Dynamic Logical Partitioning

Virtual Adapter Resources

Add / Remove as shown in Figure 4-31.

Figure 4-31 Dynamically adding or removing virtual adapters to a partition

Important: If you want to keep your dynamic virtual adapter changes after you

reactivate the partition, you also have to add or remove the defined adapters to the partition profile.

7. Run the

cfgdev

command from the command line interface of the Virtual I/O

Server to refresh the configuration from the operating system point of view.

8. Define the SEA on the Virtual I/O Server using the

mkvdev

command as follows: mkvdev -sea TargetDevice -vadapter VirtualEthernetAdapter ...

-default DefaultVirtualEthernetAdapter

-defaultid SEADefaultPVID [-attr Attributes=Value ...]

Chapter 4. Virtualized resource management

129

Using the example in Figure 4-32, the target devices are the physical

adapters (for example, ent0 and ent1). The virtual devices are ent2, ent3, and ent4, and the defaultid is the default PVID associated with the default virtual

Ethernet adapter.

Figure 4-32 Example of an I/O server partition bridge

Important: To set up the SEA, all involved virtual and physical Ethernet

interfaces have to be unconfigured (down or detached).

9. Setup SEA using the following commands:

$mkvdev –sea ent0 –vadapter ent2 –default ent2 –defaultid 1

$mkvdev –sea ent1 –vadapter ent3 ent4 –default ent3 –defaultid 2

After running the

mkvdev

command, the system creates the SEA ent5.

In the second example, the physical Ethernet adapter is ent1. The

mkvdev

command maps the virtual Ethernet adapter ent3 and ent4 to the physical adapter. Additionally, ent3 is defined as a default adapter with the default

VLAN ID of 2. Untagged packets that are received by the SEA are tagged with the VLAN 2 ID and are sent to the Virtual Ethernet adapter ent3.

10.Configure the ent5 interface with an IP address using the

mktcpip

command as shown in the following: mktcpip -hostname HostName -inetaddr Address -interface Interface

[-start] [-netmask SubnetMask] [-cabletype CableType]

[-gateway Gateway] [-nsrvaddr NameServerAddress

[-nsrvdomain Domain]]

11.Set up the hostname and IP address for the SEA as shown in the following example.

$ mktcpip -hostname p5_2ioserver1 -inetaddr 9.3.5.150 -interface en5

-netmask 255.255.255.0 -gateway 9.3.5.41

130

Partitioning Implementations for IBM

E server

p5 Servers

Restriction: Multiple subnets may connect externally using the same

Ethernet adapter; however, each subnet must be tagged with a different VLAN

ID.

4.4.6 Virtual SCSI disk

Virtual SCSI facilitates the sharing of physical disk resources (I/O adapters and devices) between logical partitions. Virtual SCSI enables partitions to access

SCSI disk devices without requiring physical resources be allocated to the partition. Partitions maintain a client/server relationship in the Virtual SCSI environment. Partitions that contain Virtual SCSI devices are referred to as client partitions while the partition that own the physical resources (adapters, devices) is the Virtual I/O Server.

The Virtual SCSI disks are defined as logical volumes or as physical volumes in the Virtual I/O Server. All standard conventional rules apply to the logical volumes. The logical volumes appear as real devices (hdisks) in the client partitions and can be used as a boot device and as a NIM target.

After a virtual disk is assigned to a client partition, the Virtual I/O Server must be available before the client partitions are able to boot.

Defining volume groups and logical volumes

If you want to create a logical volume to assign to your client partition, use the

mklv

command. To create the logical volume on a separate disk, you first have to create a volume group and assign one or more disks using the

mkvg

command.

The basic syntax of the

mkvg

command to create a volume group on the Virtual

I/O Server is: mkvg [-f] [-vg VolumeGroup] PhysicalVolume ...

The basic syntax of the

mklv

command to create a logical volume on the Virtual

I/O Server is: mklv [-mirror] [-lv NewLogicalVolume | -prefix Prefix]

VolumeGroup Size [PhysicalVolume ...]

To create a volume group and define the logical volume:

1. Create a volume group and assign a disk to this volume group using the

mkvg

command as shown. In this example the name of the volume group is rootvg_clients.

$ mkvg -f -vg rootvg_clients hdisk2 rootvg_clients

Chapter 4. Virtualized resource management

131

2. Define the logical volume which will be visible as a disk to the client partition.

The size of this logical volumes acts as the size of disks which will be available to the client partition. Use the

mklv

command to create 2 GB size logical volume called rootvg_dbsrv as follows:

$ mklv -lv rootvg_dbsrv rootvg_clients 2G rootvg_dbsrv

4.4.7 Defining the Virtual SCSI Server adapter on the HMC

To define the Virtual SCSI Server adapter on the HMC:

1. On the Virtual I/O Server partition profile select the Virtual I/O tab to create a

Virtual SCSI Server adapter. Choose SCSI and click (Create...) to proceed.

The Virtual SCSI Adapter Properties dialog box opens, as shown in

Figure 4-33.

Figure 4-33 Virtual SCSI Adapter Properties panel on the IO Server

132

Partitioning Implementations for IBM

E server

p5 Servers

The slot number identifies the virtual adapter within the logical partition. The combination of the slot number and the logical partition ID uniquely identify this slot within the managed system.

This slot number does not refer to any physical hardware location on your system. You can, therefore, assign slot numbers to virtual adapters in any way that makes sense to you, provided that you follow these guidelines:

– You can use any slot number from 2 up to (but not including) the maximum number of virtual adapters. Slots 0 and 1 are reserved for system-created virtual adapters. By default, the system displays the lowest unused slot number for this logical partition.

– You cannot use a slot number that was used for any other virtual adapter on the same logical partition.

2. Select Server as the Adapter Type.

3. In the Connection Information area, define whether or not you want to allow all partitions or only one dedicated partition to connect to this SCSI drive.

4. If the client partition is not defined yet, you can enter a unique Partition ID into the Remote Partition field. Use this Partition ID when defining your client partition. The HMC will assign this partition automatically as a client for virtual

SCSI resources. The Remote partition virtual slot number has to match with the slot number defined for your client SCSI adapter on the client partition.

5. Click OK and the Virtual SCSI Server adapter is ready to be configured from the command line interface of the Virtual I/O Server. When you define the adapter in the partition profile you have to shutdown your partition and reactivate it to make the adapter available or to use the dynamically reconfiguration process.

Chapter 4. Virtualized resource management

133

4.4.8 Defining the Virtual SCSI Client adapter on the HMC

The Virtual SCSI Client adapter is defined in the same panel in the client partition

profile. Figure 4-34 shows the client partition DB_Server definition for the Virtual

SCSI Client adapter.

Figure 4-34 Virtual SCSI Adapter Properties panel on the client partition

The Virtual SCSI Client adapter has Slot number 3 defined, which matches to the

Remote partition virtual slot number on the Virtual I/O Server partition.

The Adapter Type assigned on the client partition is Client.

In the Connection Information area, select the hosting I/O Server partition and fill in the Remote partition virtual slot number. In this example, this is slot number

20.

134

Partitioning Implementations for IBM

E server

p5 Servers

4.4.9 Creating the virtual target device on the Virtual I/O Server

The basic command to map the Virtual SCSI with the logical volume or hdisk is as follows: mkvdev -vdev TargetDevice -vadapter VirtualSCSIServerAdapter

[-dev DeviceName]

To create the virtual target device:

1. Run the

lsdev -virtual

command to make sure that the new virtual SCSI adapter is available:

$ lsdev -virtual name status description ent2 Available Virtual I/O Ethernet Adapter (l-lan)

vhost0 Available Virtual SCSI Server Adapter

vhost1 Available Virtual SCSI Server Adapter vhost2 Available Virtual SCSI Server Adapter vsa0 Available LPAR Virtual Serial Adapter

2. Create a virtual target device, which maps the Virtual SCSI Server adapter vhost0 to the logical volume rootvg_dbsrv that was created previously. When you do not use the -dev flag, the default name of the Virtual Target Device adapter is vtscsix. Run the

mkvdev

command as shown:

$ mkvdev -vdev rootvg_dbsrv -vadapter vhost0 -dev vdbsrv vdbsrv Available

To map a physical volume to the Virtual SCSI Server Adapter use hdiskx instead of the logical volume devices for the -vdev flag.

The

lsdev

command shows the newly created Virtual Target Device adapter.

$ lsdev -virtual name status description vhost0 Available Virtual SCSI Server Adapter vsa0 Available LPAR Virtual Serial Adapter

vdbsrv Available Virtual Target Device - Logical Volume

Chapter 4. Virtualized resource management

135

The

lsmap

command shows us the logical connections between newly created devices, as follows:

$ lsmap -vadapter vhost0

SVSA Physloc Client

PartitionID

--------------- --------------------------------------------

------------------

vhost0 U9111.520.10DDEEC-V1-C20 0x00000000

VTD vdbsrv

LUN 0x8100000000000000

Backing device rootvg_dbsrv

Physloc

The physical location is a combination of the slot number. In this example, 20 and the logical partition ID.

3. At this point the virtual device can be attached from the client partition. You can activate the partition with the SMS menu and install the AIX operating system on the virtual disk, or you can add an additional virtual disk using the

cfgmgr

command.

The Client PartitionID is visible as soon the client partition is active.

4.4.10 Limitations and considerations

The Virtual I/O Server software is a dedicated software only for the Virtual I/O

Server operations, and there is no possibility to run other applications in the

Virtual I/O Server partition.

There is no option to get the Virtual I/O Server partition pre-installed on new systems. At the time of the writing of this book, the preinstall manufacturing process does not allow the Virtual I/O Server partition to be pre-installed.

The Virtual I/O Server should be properly configured with enough resources. The most important are the processor resources. If a Virtual I/O Server has to host a lot of resources to other partitions, you must ensure that enough processor power is available.

Logical volume limitation

The Virtual I/O Server operating system allows you to define up to 1024 logical volumes per volume group, but the actual number that you can define depends on the total amount of physical storage that are defined for that volume group and the size of the logical volumes you configure.

136

Partitioning Implementations for IBM

E server

p5 Servers

Table 4-5 shows the limitations for logical storage management.

Table 4-5 Limitations for logical storage management

Category Limit

Volume group

Physical volume

4096 per system

1024 per volume group

Physical partition

Logical volume

Logical partition

2097152 per volume group

4096 per volume group

Based on physical partitions

Chapter 4. Virtualized resource management

137

138

Partitioning Implementations for IBM

E server

p5 Servers

Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement

Table of contents