Installing HyperFabric Adapters. HP HyperFabric

Installing HyperFabric

Installing HyperFabric Adapters

CAUTION

Installing HyperFabric Adapters

This section contains information about installing HyperFabric adapters in HP 9000 systems. Online Addition and Replacement (OLAR)

information is provided in the section, “Online Addition and

Replacement” on page 44.

HyperFabric adapters contain electronic components that can easily be damaged by small amount of electricity. To avoid damage, follow these guidelines:

• Store adapters in their antistatic plastic bags until installation.

• Work in a static-free area, if possible.

• Handle adapters by the edges only. Do not touch electronic components or electrical traces.

• Use the disposable grounding wrist strap provided with each adapter. Follow the instructions included with the grounding strap.

• Use a suitable ground—any exposed metal surface on the computer chassis.

For specific instructions, see system-specific documentation on

“installing networking adapters” for each type of HP 9000 system that

HyperFabric adapters will be installed into.

When the HyperFabric adapters have been installed, go to “Installing the

Software” on page 51.

Online Addition and Replacement

Online Addition and Replacement (OLAR) allows PCI I/O cards, adapters or controllers to be replaced or added to HP 9000 systems, without the need for completely shutting down and rebooting the system, or adversely affecting other system components. This feature is only

44 Chapter 3

NOTE

IMPORTANT

Installing HyperFabric

Installing HyperFabric Adapters available on HP 9000 systems that are designed to support OLAR. The system hardware uses the per-slot power control combined with OS support to enable this feature.

OLAR is supported only on TCP/UDP/IP over HF2 adapters.

Not all add-in cards have this capability, but over time many cards will be gaining this capability.

The HyperFabric Release Notes contains information about which

HP 9000 systems and HyperFabric adapters OLAR is supported for.

At this time, Superdome systems are not intended for access by users.

HP recommends that these systems only be opened by a qualified HP engineer. Failure to observe this requirement can invalidate any support agreement or warranty to which the owner might otherwise be entitled.

There are two methods to add or replace OLAR-compatible cards:

• Using the SAM utility.

• Issuing command-line commands, through olrad, that refer to the

HyperFabric OLAR script (/usr/sbin/olard.d/clicd).

HP recommends that SAM be used for OLAR procedures, instead of the rad command. This is primarily because SAM prevents the user from doing things that might have adverse effects. This is not true when the rad command is used.

For detailed information about using either of these two procedures, see

Configuring HP-UX For Peripherals. You can order that document from

HP, or you can view, download, and print it from the following URL: http://www.docs.hp.com.

Chapter 3 45

Installing HyperFabric

Installing HyperFabric Adapters

Table 3-1

Table 3-1 below explains some important OLAR-related terms.

Important OLAR Terms

Term Meaning

OLAR

Power Domain

All aspects of the OLAR feature including Online Addition (OLA) and Online Replacement (OLR).

A grouping of 1 or more interface card slots that are powered on or off as a unit. (Note: Multi-slot power domains are not currently supported.) target card / target card slot The interface card which will be added or replaced using OLAR, and the card slot in which it resides.

affected card / affected card slot Interface cards and the card slots they reside in, which are in the same power domain as the target slot.

IMPORTANT In many cases, other interface cards and slots within the system are dependent on the target card. For example, if the target card is a multiple-port card, suspending or deleting drivers for the target card slot also suspends individual drivers for the multiple hardware paths on that card.

During a card replacement operation, SAM performs a Critical

Resource Analysis (CRA), which checks all ports on the target card for critical resources that would be temporarily unavailable while the card is shut down.

46 Chapter 3

Chapter 3

Installing HyperFabric

Installing HyperFabric Adapters

Planning and Preparation

As mentioned previously, for the most part, SAM prevents the user from performing OLAR procedures that would adversely affect other areas of the HP 9000 system. See Configuring HP-UX For Peripherals for detailed information.

Critical Resources

The effects of shutting down a card’s functions must be considered.

Replacing a card that is still operating can have extensive consequences.

Power to a slot must be turned off when a card is removed and a new card is inserted.

This is particularly important if there is no online failover or backup card to pick up those functions. For example:

• Which mass storage devices will be temporarily disconnected when a card is shut down?

• Will a critical networking connection be lost?

A critical resource is one that would cause a system crash or prevent an operation from successfully completing if the resource were temporarily suspended or disconnected. For example, if the SCSI controller is connected to the unmirrored root disk or swap space, the system will crash when the SCSI controller is shut down.

During an OLAR procedure, it is essential to check the targeted card for critical resources, as well as the effects of existing disk mirrors and other situations where a card’s functions can be taken over by another card that will not be affected.

As mentioned earlier, SAM performs a thorough CRA automatically, and presents options based on its findings. If it is determined that critical resources will be affected by the OLAR procedure, the card could be replaced when the system is offline. If action must be taken immediately, an online addition of a backup card and deletion of the target card could be attempted using the rad command.

Card Compatibility

This section explains card compatibility considerations for doing OLAR.

47

Installing HyperFabric

Installing HyperFabric Adapters

Online Addition (OLA) Multiple cards can be added at the same time.

When adding a card online, the first issue to resolve is whether the new card is compatible with the system. Each OLAR-capable PCI slot provides a set amount of power. The replacement card cannot require more power than there is available.

The card must also operate at the slot’s bus frequency. A PCI card must run at any frequency lower than its maximum capability, but a card that could operate at only 33 MHz would not work on a bus running at 66

MHz. rad provides information about the bus frequency and power available at a slot, as well as other slot-related data.

If an HP 9000 system has one or more slots that support OLAR and OLA will be used to install a HyperFabric adapter in one of those slots—install the adapter in the HP 9000 system according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the “Configuring HP-UX Peripherals” manual.

After adding a new HyperFabric adapter, SAM tries to locate the

HyperFabric software. If SAM cannot locate the HyperFabric software, the new adapter cannot be used until the software is installed (software installation requires a system reboot). If SAM locates the HyperFabric software, SAM determines whether the new adapter is functional. If it is not functional, SAM displays an error message.

If the new adapter is functional, SAM displays a message telling the user to configure the adapter and start HyperFabric. If only one adapter is being added, issue the clic_init -c command or use SAM to configure the adapter, and then issue the clic_start command or use SAM to start HyperFabric. If multiple adapters are being added, add all of the adapters first, and then run clic_init -c and clic_start or use SAM.

See “Performing the Configuration” on page 78 and “Starting

HyperFabric” on page 109 for more information about configuring and

starting HyperFabric.

CAUTION Do not change any configuration information for an existing HyperFabric adapter or switch while you are using clic_init -c to configure a new adapter.

When you have completed the adapter installation, go to “Installing the

Software” on page 51.

48 Chapter 3

Chapter 3

Installing HyperFabric

Installing HyperFabric Adapters

Online Replacement (OLR) When replacing an interface card online, the replacement card must be identical to the card being replaced (or at least be able to operate using the same driver as the replaced card). This is referred to as like-for-like replacement and should be adhered to, because using a similar but not identical card can cause unpredictable results. For example, a newer version of the target card that is identical to the older card in terms of hardware might contain an updated firmware version that could potentially conflict with the current driver.

An A6386A adapter must be replaced with another A6386A adapter, etc.

Also, the old adapter and new adapter must have the same revision levels.

When a replacement card is added to an HP 9000 system, the appropriate driver for that card must be configured in the kernel before beginning the replacement operation. SAM ensures the correct driver is present. (In most cases, the replacement card will be the same type as a card already in the system, and this requirement will be automatically met.) Keep the following things in mind:

• If the necessary driver is not present and the driver is a dynamically loadable kernel module (DLKM), it can be loaded manually. See the

“Dynamically Loadable Kernel Modules” section in “Configuring

HP-UX For Peripherals” for more information.

• If the driver is static and not configured in the kernel, then the card cannot be added online. The card could be physically inserted online, but no driver would claim it.

If there is any question about the driver’s presence, or if it is uncertain that the replacement card is identical to the existing card, ioscan can be used together with rad to investigate.

If more than one operational HyperFabric adapter is present when SAM requests the suspend operation for all ports on the target adapter,

HyperFabric will redirect the target adapter’s traffic to a local backup adapter using local failover. Client applications using the replaced adapter will not be interrupted in any way.

If the adapter being replacing is active and it is the only operational

HyperFabric adapter on the HP 9000 system, SAM displays the following warning message:

WARNING: You have 1 operational HyperFabric card. If you go ahead with this operation you will lose network access via

HyperFabric until the on-line replaced HyperFabric card becomes operational.

49

Installing HyperFabric

Installing HyperFabric Adapters

You are asked if you want to continue. If you reply Yes, client applications are suspended. Replace the adapter according to the procedure described in the “Managing PCI Cards with OLAR” chapter of the Configuring

HP-UX Peripherals manual.

When an adapter has been replaced, client application activity resumes unless the TCP timers or the application timers have popped.

CAUTION Do not use the clic_start command or the clic_shutdown command, while an installed adapter is suspended. Do not use SAM to start or stop

HyperFabric while an installed adapter is suspended. The operation will fail and an error message will be displayed.

After a HyperFabric adapter has been replaced, SAM checks the replacement adapter to ensure that it is permitted according to the like-for-like rules. If the adapter is permitted, SAM automatically activates it. If it is not permitted, SAM displays an error message.

50 Chapter 3