EMC CLARiiON Server Support, Navisphere Host Agent, Storage System Initialization Utility, Server Utility, Command Line Interface, admsnap software Installation guide
ಜಾಹಿರಾತು
ಜಾಹಿರಾತು
EMC CLARiiON Server Support
Products for AIX Servers
Installation Guide
P/N 300–002–044, Revision A13
October 3, 2008
This guide describes how to install and remove the EMC® Navisphere®
Host Agent, Navisphere Storage System Initialization Utility, Navisphere
Server Utility, Navisphere Command Line Interface (CLI), and admsnap software on an IBM System p server with the AIX operating system.
If you have an RS 6000 SP server, refer to revision A08 of this guide, which is available on the
Powerlink
® website.
For information on supported operating system versions and server software with a CX4 series, CX3 UltraScale™series, or a CX series storage system, refer to the E-Lab™ Interoperability Navigator on the
Powerlink
® website. For AX4–5 series storage systems, refer to the Support Matrix link on the Install page of the storage-system support website.
Topics include:
Storage system terminology ........................................................... 3
About EMC Navisphere server software ......................................... 4
Software for your storage system.................................................... 7
Finding current information ........................................................... 8
Installing and removing the Navisphere Storage System Initialization
Utility .......................................................................................... 9
Running the Navisphere Storage System Initialization Utility ........... 12
Determining whether to install the host agent or server utility
Requirements for the host agent, server utility, and CLI ................... 20
Installing or removing the Navisphere Host Agent and CLI ............. 21
Configuring the Navisphere Host Agent ........................................ 27
Using the Event Monitor configuration file ..................................... 29
Running the Navisphere Host Agent ............................................. 30
Installing and removing the Navisphere Server Utility .................... 33
1
Running the Navisphere Server Utility ........................................... 36
Installing and removing the Admsnap Utility ................................. 41
Running the Admsnap Utility ........................................................ 45
2
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Storage system terminology
This document uses the following storage system terms:
CX4 series — refers to CX4-120, CX4-240, CX4-480, and CX4-960 storage systems
CX3 series — refers to CX3 model 10 systems, CX3 model 20 systems, CX3 model 40 systems, and CX3 model 80 storage systems
CX series — refers to CX300 series, CX500 series, and CX700 storage systems
AX4-5 series — refers to AX4-5SC, AX4-5SCi, AX4-5, AX4-5i storage systems
AX150 series — systems include the AX150SC, AX150SCi, AX150, and AX150i. AX100 series systems include the AX100SC, AX100SCi,
AX100, and AX100i
AX series — to refer to AX150 series and AX100 series storage systems
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
3
About EMC Navisphere server software
The EMC server-based software listed below is available on the server support CD that shipped with your storage system.
For information on the server software supported with your
CLARiiON® storage system and the minimum FLARE® version
required, refer to Software for your storage system, page 7 .
Navisphere Storage System Initialization Utility
Navisphere Host Agent
Navisphere Server Utility
Admsnap Utility
Navisphere Command Line Interface (CLI)
About the Navisphere Storage System Initialization Utility
For Fibre Channel storage systems, use the Navisphere
® Initialization utility to discover storage systems and set network parameters (IP address, subnet mask, and default gateway). In addition, for iSCSI storage systems with iSCSI data ports attached to Windows servers, use the utility to set network parameters for these ports.
For CX series storage systems, an authorized service provider must install and run the initialization utility.
You can run the initialization utility from the server support CD or you can install it on a server or other host such as a Navisphere off-array management station. The only requirement is that the host be connected to the same network subnet as the storage-system management ports.
About the Navisphere Host Agent
The host agent registers the server’s host bus adapter (HBA) with the attached storage system when the host agent service starts. This action sends the initiator records for each HBA to the storage system. Initiator records are used to control access to storage-system data.
The agent can then retrieve information from the storage system automatically at startup or when requested by Manager or CLI.
4
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
The host agent can also:
Send operating system drive mapping information to the attached
CLARiiON® storage systems.
Monitor storage-system events and notify personnel by email, page, or modem when any designated event occurs.
Retrieve LUN world wide name (WWN) and capacity information from Symmetrix® storage systems.
About the Navisphere Server Utility
The server utility allows you to perform the following functions:
Update server information to the storage system — Lets you send the server name and IP address to the storage system and, if needed later on, allows you to update or view this data.
Verify server high availability (HA) — Lets you determine if the server is configured for high availability (HA) by verifying that the server has at least one connection path to each storage-system
SP, and that PowerPath® or some other failover software, such as
VMware native, DMP, PV Links, or HP native failover for 11iV3 is running. The utility does not detect any other native failover software, such as Sun StorEdge Traffic Manager or Linux native multipath (MPIO).
This feature is not available on iSCSI servers.
Use the snapshot feature — On storage systems with Navisphere
Express, you can start and stop a snapshot on the source server
(server assigned to the source virtual disk or source LUN), or you can allow or remove access to the snapshot by the secondary server
(server assigned to the snapshot).
You can run the Navisphere Server Utility from the CD or you can install it on the servers that are connected to the storage system. We strongly recommend that you install the utility on each server that is connected to the storage system.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
5
On AX4–5 series storage systems with Navisphere Manager, you cannot use the server utility to manage snapshots. Instead, you must install and use the
admsnap
utility.
About the Admsnap Utility
The Admsnap Utility is an executable program that you can run interactively or with a script to manage SnapView® clones and snapshots. The Admsnap Utility resides on the servers connected to the storage system with the SnapView driver installed.
Use admsnap commands to:
Scan for new storage devices
Make storage devices inaccessible to the server operating system
Flush cache data to disk
List current snapshot devices
Map and unmap to a SnapView session
Start and stop a SnapView session
About the Navisphere CLI
The CLI complements, or is an alternative to Manager. It provides a command line interface for storage-system management, including storage provisioning, status and configuration information retrieval, and control. You can use the CLI to automate management functions through shell scripts and batch files. For more information, refer to the
EMC Navisphere Command Line Interface (CLI) Reference Guide.
6
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Software for your storage system
Table 1 lists the minimum FLARE version that your CLARiiON storage
system requires and the server software that it supports.
IMPORTANT
AX150 series and AX100 series storage systems are not supported on an
AIX attached server.
Storage system
AX4-5 series
AX4-5 series
AX150 series
AX100 series
Table 1
CX4–120, CX4–240, CX4–480, and CX4–960
CX3–10c
CX3 model 20 systems, CX3 model 40 systems, and CX3 model 80 systems
CX700, CX500 series, and
CX300 series
Minimum FLARE versions and supported software
Minimum FLARE version
02.23.050.5.5xx (Navisphere
Express or Navisphere Manager)
02.23.050.5.xxx (Navisphere
Express only)
02.20.xxx.5.yyy
02.16.xxx.5.yyy
04.28.000.5.xxx
03.24.010.5.xxx
03.22.xxx.5.yyy
02.16.xxx.5.yyy
Supported server software
Storage systems with Navisphere Express
— Navisphere Storage System Initialization Utility
— Navisphere Server Utility
— Navisphere CLI
Storage systems with Navisphere Manager
— Navisphere Storage System Initialization Utility
— Navisphere Host Agent
— Navisphere Server Utility
— Admsnap Utility
— Navisphere CLI
Storage systems with Navisphere Manager
— Navisphere Storage System Initialization Utility
— Navisphere Host Agent
— Navisphere Server Utility
— Admsnap Utility
— Navisphere CLI
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
7
Finding current information
The most up-to-date information about the CLARiiON® server software is posted on the EMC Powerlink® website. We recommend that you download the latest information before you install any server software.
To access EMC Powerlink, use the following link:
http://Powerlink.EMC.com
After you log in, select Support > Technical Documentation and
Advisories
and find the following:
Navisphere Host Agent/CLI and Utilities Release Notes.
EMC SnapView Command Line Interfaces Reference (P/N 069001181).
Installation Roadmap for CLARiiON Storage Systems (P/N 069001166), which provides a checklist of the tasks that you must complete to install your storage system in a storage area network (SAN) or direct attach configuration.
For the most current management and security content, refer to the
EMC Navisphere Manager help.
For Navisphere CLI commands, refer to the latest revision of the EMC Navisphere Command Line Interface (CLI) Reference (P/N
300–003–628).
8
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Installing and removing the Navisphere Storage System Initialization
Utility
EMC recommends that you install the utility on the server; however, you can run the Navisphere Storage System Initialization Utility from the server support CD.
Installation prerequisites
To initialize a storage system using the Navisphere Storage System
Initialization Utility, the server must meet the following requirements:
Run a supported version of the operating system.
Be connected to the same subnet as the 10/100 management ports of the storage system that you want to initialize. This server may also be a server with Fibre Channel connections to the storage system.
If you have an earlier version of the storage system initialization utility installed, you must uninstall it before installing an updated version.
Installing the Navisphere Storage System Initialization Utility on a Windows host
EMC recommends that you install the most recent version of the
Navisphere Storage System Initialization Utility software that is appropriate for your configuration. You can download the most recent version from the software download page on the Powerlink website (CX4 series, CX3 series, CX series, and AX4-5 series) or on the storage-system support website (AX series). You can also install the software from the server support CD (any storage system); however, the CD may not contain the most recent version for your configuration.
1. Log in as the administrator (or someone who has administrative privileges) to the Windows host that is on the same subnet as the storage system.
2. To download the software for CX4 series, CX3 series, CX series, or
AX4–5 series storage systems, do the following: a. From the Navigator drop-down menu at the top right of the
Powerlink
website, select Software Downloads and Patches.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
9
b. Select the appropriate Navisphere Storage System Initialization
Utility version to download and select the option to save the software to your server.
c. In the directory where you saved the software, double-click the executable file to start the installation wizard.
3. To install the software from the server support CD (any storage system), do the following: a. In the drive of Windows host, insert the server support CD, which shipped with the storage system or the upgrade kit (for
AX4-5 series or AX series systems upgrading to Navisphere
Manager).
The server support menu opens. If you do not see the server support menu, follow the steps below to open it.
From the Windows taskbar, select:
Start > Run
In the Run dialog box, enter:
drive:\locales\en_US\start.html where drive is the letter for the CD drive. Click OK.
b. Select your language, if prompted for it.
c. From the main menu, click Install Products on Server.
d. From the Install Products menu, click Navisphere Storage
System Initialization Utility
to open the installation wizard.
4. Follow the instructions on the installation screens to install the initialization utility.
A user interface (UI), a text-based, and a command line version of the utility are installed.
5. When the installation is complete, click Done to exit the wizard.
10
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
6. If you installed the initialization utility from the CD, close the server support menu by selecting Main Menu and then Exit. You can now remove the CD from the server’s CD drive.
Removing the Navisphere Storage System Initialization Utility on a Windows host
1. On the Windows host running the Storage System Initialization
Utility that you want to remove, log in as the administrator or the equivalent.
2. Open Control Panel by selecting Start > Settings > Control Panel or Start > Control Panel.
3. On the Control Panel pane, double-click Add/Remove Programs or
Program and Features
.
4. Select Navisphere Storage System Initialization Utility and click
Change/Remove
or Uninstall/Change.
5. If asked to confirm the removal, click OK and follow the steps to remove the software program.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
11
Running the Navisphere Storage System Initialization Utility
You can start the storage system initialization utility from the server on which it is installed or from the server support CD. The storage system initialization utility has three methods for executing commands — a user interface (UI) version, a command line interface that can be scripted, or a text-based utility that cannot be scripted.
Before continuing, make sure that the storage system is powered up completely.
For storage-system initialization, connect the server from which you are running the utility and the storage system to the same network subnet. Once you have assigned an IP address to the storage system, the server and storage system can be on different subnets.
Starting the initialization utility UI on a Windows host
To start the UI version of the initialization utility from the server, select Start > Programs > EMC > Navisphere > Navisphere Storage
System Initialization
or Start > All Programs > EMC > Navisphere
> Navisphere Storage System Initialization
.
Starting the text-based or command line initialization utility on a Windows host
To start the text-based version, at a command prompt enter cd
C:\Program Files\EMC\Navisphere Storage System Initialization
Wizard
or cd C:\Program Files (x86)\EMC\Navisphere Storage
System Initialization Wizard
; then enter NaviInitToolCLI.exe.
To start the command line version, at a command prompt enter cd
C:\Program Files\EMC\Navisphere Storage System Initialization
Wizard
or cd C:\Program Files (x86)\EMC\Navisphere Storage
System Initialization Wizard
; then enter ./naviinittoolcli and any
12
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
of the command line switches in Using the command line initialization
Starting the initialization utility on a Windows host from the CD
1. Log in to the Windows server as the administrator or someone who has administrative privileges.
2. In the server’s drive, insert the server support CD, which shipped with the storage system or the upgrade kit (for AX4-5 series or AX series systems upgrading to Navisphere Manager). The server support menu opens. If you do not see the server support menu, follow these steps to open it:
From the Windows taskbar, select:
Start > Run
In the Run dialog box, enter:
drive:\locales\en_US\start.html where drive is the letter for the CD drive. Click OK.
3. From the main menu select Run Products from CD. The RUN
PRODUCTS
page opens.
4. To start the utility, select Navisphere Storage System Initialization
Utility
.
Using the UI version of the initialization utility
After you start the utility, it automatically scans the subnet for supported storage systems. When this storage-system discovery operation is complete, the utility lists, by hardware serial number, all uninitialized and initialized storage systems it found. The storage-system serial number is on a blue label located on the SPE chassis behind the front bezel.The hardware serial number is on a blue label on the rear of the chassis.The hardware serial number is on a hanging tag at the rear of the storage system.
If the discovery operation did not find the storage system you are installing, verify that the storage system’s management ports are properly cabled to the LAN on which the server resides. The management ports and the server must both reside on the same subnet.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
13
Proceed through the wizard screens, providing information where required. For more information about completing the wizard, click
Help
in the wizard’s menu bar.
For a CX4 and CX3 series storage systems
If you entered an SP IP address for the first time or changed an SP IP address, the utility reboots the storage system, and the SP fault light on the back of each SP starts blinking. The reboot takes several minutes to complete. When it has completed, the SP fault light (LEDs) on each
SP stops blinking and remains off.
For an AX4-5 series storage system
If you entered an SP IP address for the first time or changed an SP IP address, the utility reboots the storage system, and green lights flash to indicate activity. The reboot takes 5 to 6 minutes to complete. When it has completed, the power button on each SP remains solid/steady green.
Using the command line initialization utility
After you start the command line version of the initialization utility, use any of the switches below with the naviinittoolcli .
To start the command line version of the initialization utility, at a command prompt enter cd C:\Program Files\EMC\Navisphere Storage System
Initialization Wizard
or cd C:\Program Files (x86)\EMC\Navisphere Storage
System Initialization Wizard
; then enter ./naviinittoolcli.
eula
[-language language]
discover
[-all] [-xml]
configure
–serial serialnumber [-file filename] [–ipa
spAIpaddress|–sphosta spAhostname] [-ipb spBIpaddress
|–sphostb spBhostname] [–mask subnetmask] [–gateway gatewayaddress]
[-user username] [–password password] [–storagename storageName]
[-ipv6mode |automatic|disable| [manual [-globalprefix prefix
[-ipv6gateway ipv6gateway]] [ -help] where:
14
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
eula
Displays the EMC end-user license agreement (EULA) in the specified language.
-language language The default language is English.
Currently, the only valid language is English.
discover
Discovers and displays a list of partially initialized storage systems.
-all Discovers and displays a list of initialized and partially initialized storage systems.
-xml Specifies that the output is in xml format.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
15
configure
Specifies the network parameters for the specified storage system.
-serial serialnumber
-file filename
Specifies the serial number of the storage system you want to initialize.
Specifies the name of the file that will store all the network parameters.
-ipa spAipaddress |-sphosta spAhostname
-ipb spBipaddress |-sphostb spBhostname
-mask subnetmask
-gateway gateway
-user username
Specifies the IP address of SP A, or the server name for SP A for the specified storage system.
Specifies the IP address of SP B, or the server name for SP B for the specified storage system.
Specifies the subnet mask for the specified storage system.
Specifies the gateway for the specified storage system.
Specifies the storage-system login username.
-password password
-storagename storagename
-help
-ipv6mode automatic | disable |
[manual –globalprefix prefix
–ipv6gateway ipv6gateway]
Specifies the storage-system login password.
Specifies the name for the specified storage system.
Displays the help screen and does not start the initialization process
Specifies the configuration type for the IPv6 network. Refer to the IPv6 configuration type table below for details. IPv6 is supported with CX4 series systems only.
16
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Table 2 IPv6 parameter values based on IPv6 configuration type (CX4 series systems only)
Existing configuration type
(What you see when you open the initialization utility)
New configuration type
(What you see after changing the configuration type)
Disabled
Auto
All IPv6 parameteers are blank and grayed out.
IPv4 will be used
Current IPv6 parameters are visible but not editable.
Disabled
All IPv6 parameters are blank and grayed out *
IPv4 will be used
Auto
All IPv6 parameters are set to
TBD*
Manual
Link Local:
TBD*
Global IP:
Global
Prefix:
TBD*
Blank/editable
Gateway:
blank/editable
Link Local:
visible**
Global IP:
Global
Prefix:
Gateway:
visible** visible/editable** visible/editable**
Manual Link Local:
visible
Global IP:
visible
Global
Prefix:
Gateway:
visible/editable visible/editable
All IPv6 parameters are blank and grayed out * IPv4
will be used
Current IPv6
Link Local parameter is visible. All others are set to TBD **
* After changing the configuration type from Disabled to Auto or Manual, or from Auto or Manual to Disabled, the new values will be set after the automatic reboot of the storage system. After the reboot completes, open the initialization utility again to view the new IPv6 values.
** After changing the configuration type from Manual to Auto or from Auto to Manual, a reboot is not required.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
17
Determining whether to install the host agent or server utility
Depending on your application needs, you can install the host agent, server utility, or both on an attached server.
If you want to install both applications, you must install version 6.22.20
or later of the server utility either before or after you install the host agent. The registration feature of the server utility will be disabled and the host agent will be used to register the server’s NICs or iSCSI
HBAs to the storage system.
Prior to version 6.22.20 of the server utility, you could not install the host agent and server utility on the same server. However, you could install them on different servers that are connected to the same storage system.
If you do not want to install both applications, but want to use some of the features available in the server utility, you can run the server utility from the server support CD.
The host agent is not supported with AX4-5 series storage systems with
Navisphere Express; you can only use the server utility.
Table 3 describes the differences between the host agent and the server
utility on a server attached to a storage system.
Depending on the version of the server utility, some of the features listed in
may not be available.
Function
Pushes LUN mapping and OS information to the storage system.
Table 3 Differences between the host agent and the server utility
Host agent
Yes – LUN mapping information is displayed in the Manager UI next to the LUN icon or with the CLI using the -lunmapinfo command.
Server utility
AX4–5 series storage systems
Yes – LUN mapping information is displayed in the
Manager UI next to the LUN icon, or with the CLI using the -lunmapinfo command. In Navisphere Express this information is displayed in the Manage Virtual Disks page.
Note: The text
Manually Registered appears next to the hostname icon in the Manager UI indicating that the host agent was not used to register this server.
18
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Function Host agent Server utility
CX4 series, CX3 series, and CX series storage systems
No – LUN mapping information is not sent to the storage system. Only the server’s name, ID, and IP address are sent to the storage system.
Note: The text
Manually Registered appears next to the hostname icon in the Manager UI indicating that the host agent was not used to register this server.
Runs automatically to send information to the storage system.
Yes – No user interaction is required.
AX4–5 series storage systems
No – You must manually update the information by starting the utility or you can create a script to run the utility. Since you run the server utility on demand, you have more control as to how often or when the utility is executed.
CX4 series, CX3 series, and CX series storage systems
No – You must manually update the information by starting the utility or you can create a script to run the utility. Since you run the server utility on demand, you have more control as to how often or when the utility is executed.
Provides high-availability
(HA) validation.
Requires network connectivity to the storage system.
No – You can run the server utility’s
HA validation feature from the CD without having to install the server utility.
Yes – Network connectivity allows
LUN mapping information to be available to the storage system.
AX4–5 series storage systems
Currently not supported.
CX4 series, CX3 series, and CX series storage systems
Yes – The HA validation feature verifies that failover software is installed on the server and that working paths exist between the server and each SP in the storage system.
AX4–5 series storage systems
No – LUN mapping information is sent through a data path to the storage system.
CX4 series, CX3 series, and CX series storage systems
No – LUN mapping information is not sent to the storage system.
Note that if you are using the server utility to upload a high availability report to the storage system, you must have network connectivity.
Requires installation.
Yes – You must install the host agent on the server.
AX4–5 series storage systems
No – You can run the server utility from the CD.
However, we recommend that you install it on the server.
CX4 series, CX3 series, and CX series storage systems
No – You can run the server utility from the CD. However, we recommend that you install it on the server.
Note: If you install the host agent and then version 6.22.20 or later of the server utility, the server utility’s registration service feature will not be installed. Prior to version 6.22.20 of the server utility, you could not install the host agent and server utility on the same server.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
19
Requirements for the host agent, server utility, and CLI
To run the host agent, CLI, or server utility, your server must meet the following requirements:
Run a supported version of the AIX operating system.
For Fibre Channel connections, have the EMC CLARiiON supported HBA hardware and driver installed.
Be connected to at least one SP (two SPs for high-availability) in each storage system either directly or through a switch or hub. Each
SP must have an IP connection.
For the host agent and CLI only - Be on a TCP/IP network connected to at least one SP (two SPs for high-availability) in the storage system. The TCP/IP network connection allows the server to send
LUN mapping information to the storage system and it allows
Navisphere Manager or CLI to communicate with the storage system over the network.
Have a configured TCP/IP network connection to any remote hosts that you will use to manage the server’s storage systems, including: z z any server whose browser you will use to access Manager 6.X.
any Windows Server 2008, 2003, or Windows 2000 server running Storage Management Server software.
z any AIX, HP-UX, Linux, VMware ESX Server, NetWare, Solaris,
Windows Server 2003, or Windows 2000 server running the CLI.
If you want to use the CLI on the server to manage storage systems on a remote server, the server must be on a TCP/IP network connected to both the remote server and each SP in the remote server’s storage system. The remote server can be running AIX, HP-UX, Linux, Solaris, or the Windows operating system.
For information about the specific revisions of the server operating system and the storage-system FLARE and Access Logix software that are required for your version of the host agent, see the release notes for the host agent
on the EMC Powerlink website. To access this website, see Finding current
.
20
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Installing or removing the Navisphere Host Agent and CLI
You can install both the host agent and CLI, or you can install them separately.
To modify an existing host agent configuration refer to the next section.
If you want to install the host agent and use some of the features in the server
utility, see Determining whether to install the host agent or server utility , page 18
Installing the host agent and CLI on an AIX server
To install the Navisphere agent and/or CLI, you must:
Determine whether a previous version of the software you want to install (host agent and/or CLI) is currently installed on the host and, if so, remove it.
Install the Navisphere agent files and/or the CLI files. Before installing the host agent, make sure that the disk-array storage system is connected to the AIX server on which you are installing
Navisphere agent.
Determining if an earlier version is installed
1. On the AIX server, log in as superuser (root).
2. Open an xterm window and enter the following command to list currently installed Navisphere software:
lslpp -L | grep NAV
3. Examine the list of software to determine if an earlier version of
Navisphere agent or CLI is present. If a previous version is present, remove it before continuing to the next section.
Installing the agent and/or CLI files
EMC recommends that you install the most recent version of the
Navisphere Host Agent/CLI software that is appropriate for your configuration. You can download the most recent version from the
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
21
22
software download page on the Powerlink website. You can also install the software from the server support CD; however, the CD may not contain the most recent version for your configuration.
Use the following procedure to install the host agent, the CLI, or both the host agent and CLI:
1. To download the software for CX4 series, CX3 series, CX series, or
AX4–5 series storage systems, do the following: a. From the Navigator drop-down menu at the top right of the
Powerlink
website, select Software Downloads and Patches.
b. Select the appropriate Navisphere Host Agent/CLI version and download the zip file to a Windows host.
c. Unzip the zip file you downloaded.
d. Log in to the AIX server as root.
e. Transfer the .lpp file to the /usr/sys/inst.images folder on an
AIX server.
2. To install the software from the server support CD, insert the CD into the host’s CD drive and use the following command to mount the CD:
mount -v cdrfs -o ro /dev/cd0 /cdrom
3. If the SMIT (System Management Interface Tool) System
Management
is not already open, enter the following command to open it:
smit
4. From the SMIT System Management window, follow the path:
Software Installation and Maintenance > Install and Update
Software > Install and Update from ALL Available Software
The Install and Update from ALL Available Software window opens.
5. In the Install and Update from ALL Available Software window, either select List and then the directory that contains the Navisphere software, or enter the directory path in the Input device/directory
for Software
field.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
6. Click OK and at the confirmation prompt and click OK again.
7. On the line, SOFTWARE to install, select List .
8. From the list that appears, select the Navisphere software that you want to install (NAVIAGENT or NAVICLI or both), and click OK.
For AX4–5 storage systems with Navisphere Express, you can only install the Navisphere CLI; the host agent is not supported.
9. Click OK in the ALL Available Software window.
10. When the confirmation window is displayed, click OK.
The installation process begins and the progress of the installation appears in the Install and Update from ALL Available Software window. When the installation completes, the same window indicates that the installation was successful.
You may need to scroll down to the bottom of the output window to view the Installation Summary.
11. Using SMIT, verify that the Navisphere Agent and/or CLI software are installed.
12. Configure the LUNZ device for AIX using the cfgmgr command.
13. If you have a multihomed host (multiple NIC cards installed on the host), create an agentID.txt file to ensure that the agent finds the correct NIC card.
A multihomed host machine has multiple IP addresses connected to networks; it is physically connected to multiple data links that can be on the same or different networks. When you install Navisphere Host Agent on a multihomed host, the host agent, by default, binds to the first NIC in the host. If your host is multihomed, for the host agent to successfully register with the desired CLARiiON storage system, you need to configure the host agent to bind to a specific NIC. You do this by setting up an agentID.txt file.
a. Using a text editor that does not add special formatting, create or edit a file named agentID.txt (case sensitive) in either / (root) or in a directory of your choice.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
23
b. Add the following two separate lines to the agentID.txt file, with no special formatting:
First line:
Fullyqualified hostname of the host
Second line:
IP address of the HBA/NIC port that you want Navisphere agent to use
For example, if your host is named host28 on the domain
mydomain.com
and your host contains two HBAs/NICs,
HBA1/NIC1 with IP address 192.111.222.2 and HBA2/NIC2 with IP address 192.111.222.3, and you want the Navisphere agent to use NIC 2, you would configure agentID.txt as follows:
host28.mydomain.com
192.111.222.3
The agentID.txt file should contain only these two lines, without any formatting. The first line should contain the hostname and the second line should contain the IP address of the HBA/NIC port as described above. Both lines should end with a carriage return. As a result, the cursor will be on a third line but this line should not contain any text.
c. Save the agentID.txt file.
d. If you created the agentID.txt file in a directory other than root, for host agent to restart after a system reboot using the correct path to the agentID.txt file, set the environment variable
EV_AGENTID_DIRECTORY
to point to the directory where you created agentID.txt.
e. If a HostIdFile.txt file is present in the directory shown for your operating system, delete or rename it. The HostIdFile.txt file is located in /etc/log/HostIdFile.txt.
f.
Stop and then restart the host agent.
g. Once the host agent has restarted, verify that the host agent is using the IP address that is entered in the agentID.txt file. To do this, view the new HostIdFile.txt file. You should see the IP address that is entered in the agentID.txt file.
24
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
What next?
If you installed the host agent and you are setting up a new server, go
to Running the Navisphere Host Agent , page 30. If you are just upgrading
the host agent or CLI on an existing system, or if you are just installing the CLI, you have finished the installation.
Removing the Navisphere Host Agent and/or CLI on an AIX server
To avoid losing site-specific information, copy the Navisphere agent configuration file (agent.config) to a safe location before removing any
Navisphere software.
1. On the AIX server, log in as superuser (root).
2. At the AIX shell prompt in an xterm window, start SMIT:
smit
The SMIT System Management window opens.
3. From the SMIT System Management window, follow this path:
Software Installation and Maintenance > Software
Maintenance and Utilities > Remove Installed Software
The Remove Installed Software window opens.
4. In the Remove Installed Software window, click the List button for Software name.
5. From the list that appears, select the Navisphere software you want to remove (NAVIAGENT, NAVICLI, or both), and click OK.
6. In the Remove Installed Software window, click the List button for
PREVIEW only?
(The remove operation will NOT occur).
7. From the list that appears, click no.
8. In the Remove Installed Software window, click the List button for
REMOVE dependent software?
9. From the list that appears, click yes.
10. In the Remove Installed Software window, click OK.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
25
11. When the confirmation window is displayed, click OK. The files are removed. The removal process is complete when you see a success message.
26
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Configuring the Navisphere Host Agent
Before you can use the Navisphere CLI navicli register command to configure your storage system, you must make sure the host agent
configuration file includes a privileged user, as described in Adding
The pathname of the host agent configuration file is /etc/Navisphere/agent.con-
fig
.
If you are configuring a legacy storage system, refer to revision A08 of this guide.
A legacy system is an FC series storage system that is not running storage management server software.
Adding privileged users
If you will use Navisphere CLI to configure any storage system, the host agent configuration file must include an entry that defines the person who will issue the CLI commands as a privileged user.
To define a privileged user, add a local or remote privileged user by adding the appropriate entry below.
For a local user:
user
name
For a remote user:
user
name@hostname
where name is the person’s account name and hostname is the name of the remote server the person will be using.
The default host agent configuration file includes a user root entry.
Saving the host agent configuration file
1. Save the host agent configuration file.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
27
2. If you have finished adding information to the host agent configuration file, stop and restart the host agent by entering the following commands:
rc.agent stop rc.agent start
28
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Using the Event Monitor configuration file
The Navisphere Host Agent can monitor storage-system events and take such action as sending email or paging you if specified events occur. The Event Monitor that ships with Navisphere Manager provides an interactive way to define these events and actions. If you do not have Event Monitor, you can still define such events and actions by editing the Event Monitor configuration file.
/etc/Navisphere/Navimon.cfg
The file is self-documenting; that is, text in it describes how to define events and the actions you want taken if the events occur. You can test the file after editing it using the Navisphere CLI command
responsetest
, as explained in the Navisphere Command Line Interface
Reference.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
29
Running the Navisphere Host Agent
This section describes how to test the host agent connections and how to start and stop the host agent.
Starting and stopping the Navisphere Host Agent
The host agent starts automatically when you bring the server up to init level 2. When you first start the host agent, look at the system log for the server’s operating system to make sure the agent started and no device errors occurred.
When a storage system experiences heavy input/output traffic (that is, applications are using the storage system), information may not be reported to the host agent in a timely manner, resulting in the host agent taking several minutes to execute a storage-system management task. This behavior is most evident when one host agent is managing multiple storage systems. Also, if the
SP event log is large and the host agent configuration file is set up to read all events, it may take a few minutes for the host agent to start.
Starting the host agent
Log in as root and enter this command:
rc.agent start
Stopping the host agent
Log in as root and enter this command:
rc.agent stop
Testing the host agent connections
Before continuing, you should test the host agent connections as follows:
1. Start the host agent as described in Starting and stopping the
Navisphere Host Agent, page 30.
2. Look for any errors on the console and in the operating system log to make sure the agent started and no device errors occurred.
30
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
3. Verify that the host agent on the server can see the storage system as follows: a. Enter the following CLI command:
naviseccli
|navicli [-d device]| -h hostname port
-list -hba
You cannot specify both the –d switch and –h switch.
where:
[-d device] is the device name for the storage system (only supported with legacy systems).
-h
hostname is the IP address of the SP.
For each HBA in the server, a listing similar to the following will be displayed. For storage systems in a SAN (shared storage) environment, the listing includes HBAs in all connected hosts.
Information about each HBA:
HBA UID: 10:00:00:60:B0:3E:46:AC:10:00:00:60:B0:3E:46:AC
Server Name: siux134
Server IP Address: 128.221.208.134
HBA Model Description:
HBA Vendor Description:
HBA Device Driver Name:
Information about each port of this HBA:
SP Name: spa
HBA Devicename: sp0
Trusted: NO
Logged In: YES
Source ID: 1
Defined: YES
Initiator Type: 0
Storage Group Name:
Storage Group 134 b. In the listing, examine the information for each HBA installed in the host to verify the path from the HBA to the SP.
Host agent status and error logging
While the system is running, the operating system tracks information about host agent events and host agent errors, and places this information in log files on the server.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
31
The host agent error log tracks information about the host agent’s startup, the host agent shutdown, and errors that might occur, such as the host agent’s inability to access a device in the configuration file. If problems occur, log files are a good place to start your troubleshooting.
Host agent events and errors are logged in /etc/log/agent.log.
32
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Installing and removing the Navisphere Server Utility
This section describes how to install the Navisphere Server Utility.
Installing the Navisphere Server Utility on an AIX server
EMC recommends that you install the most recent version of the
Navisphere Server Utility software that is appropriate for your configuration. You can download the most recent version from the software download page on the Powerlink website. You can also install the software from the server support CD; however, the CD may not contain the most recent version for your configuration.
Determining if an earlier version is installed
To determine if an earlier version is installed:
1. At the AIX server, log in as superuser (root).
2. Open an xterm window and enter the following command to list currently installed Navisphere software:
lslpp -L | grep HOST
3. Examine the list of software to determine if an earlier version of Navisphere Server Utility is installed. If an earlier version is installed, remove it before continuing.
Installing the Navisphere Server Utility
EMC recommends that you install the most recent version of the
Navisphere Server Utility software that is appropriate for your configuration. You can download the most recent version from the software download page on the Powerlink website. You can also install the software from the server support CD; however, the CD may not contain the most recent version for your configuration.
1. Log in to the AIX server as the administrator or someone who has administrative privileges.
2. To download the software from the
Powerlink
website for your storage system, do the following: a. From the Navigator drop-down menu at the top right of the
Powerlink
website, select Software Downloads and Patches.
b. Select the appropriate Navisphere Server Utility version and download the zip file to a Windows host.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
33
34
c. Unzip the zip file you downloaded.
d. Transfer the .lpp file to the usr/sys/inst.images folder on an AIX server.
3. To install the software from the server support CD (any storage system), insert the CD into the server’s CD drive and use the following command to mount the CD:
mount -v cdrfs -o ro /dev/cd0 /cdrom
4. Change the current directory to the server or CD directory with the HOSTUTIL software.
5. If you are not in the SMIT (System Management Interface Tool)
System Management window, open this window by entering the following command:
smit
6. From the SMIT System Management window, follow this path:
Software Installation and Maintenance > Install and Update
Software > Install and Update from ALL Available Software
The Install and Update from ALL Available Software window opens.
7. In the Install and Update from ALL Available Software window, select List and then the directory that contains the HOSTUTIL software.
8. Click OK in the ALL Available Software window.
9. When the confirmation window is displayed, click OK. The installation process begins and the progress of the installation appears in the Install and Update from ALL Available Software window.
The installation is complete when you see an installation summary message.
10. If you installed the software from the CD, unmount the CD and then remove the CD from the server’s CD drive:
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
unmount /cdrom
Removing the server utility on an AIX server
1. At the root directory, start SMIT by entering:
smit
The SMIT System Management window opens.
2. From the SMIT System Management window, follow this path:
Software Installation and Maintenance > Software
Maintenance and Utilities > Remove Installed Software
The Remove Installed Software window opens.
3. In the Remove Installed Software window, click the List button for
Software name
.
4. From the list that appears, select HOSTUTIL, and click OK.
5. Click the List button for
PREVIEW only?
(remove operation will NOT occur)
.
6. From the list that appears, click no.
7. Click the List button for
REMOVE dependent software
.
8. From the list that appears, click yes.
9. Then click OK.
10. When the confirmation window appears, click OK. The files are removed.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
35
Running the Navisphere Server Utility
This section describes how to start and use the server utility. You can run the server utility from the server or from the server support CD.
Starting the Navisphere Server Utility on an AIX server
1. Open a shell window.
2. Navigate to the HOSTUTIL directory and run the server utility:
/usr/lpp/HOSTUTIL/naviserverutil
Starting the Navisphere Server Utility on an AIX server from the CD
1. In the server’s drive, insert the server support CD that shipped with your storage system.
2. At the AIX shell prompt, create a CD directory:
mkdir /cdrom
3. Mount the CD drive:
mount -v cdrfs -p -r /dev/cd0 /cdrom
4. Navigate to the following AIX directory:
cd /cdrom/aix/usr/sys/inst.images
5. Run the server utility:
./naviserverutil
Using the Navisphere Server Utility
The Navisphere Server Utility allows you to perform the following functions:
Update server information on the storage system
Generate a high-availablity report (not available on servers with iSCSI connections)
36
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Updating server information on the storage system
The Update Server Information option allows you to send information about the server to all connected storage systems. To update server information, select option 1 from the server utility’s Welcome screen, and follow the instructions on the utility.
Generating a high-availability report
The Generate high availability report option (option 3) and the
naviserverutilcli hav
command determine if the server is configured for high availability by verifying that the server has at least one connection path to each storage-system SP, and that PowerPath or some other failover software, such as DMP is running.
After the verification, the utility generates a summary report and saves it to the server. If you issue the -upload switch with the
naviserverutilcli hav
command, the utility also sends the report to the specified storage system.
Generate a high-availability report to:
check the status of a server.
prepare for a software installation on a storage system.
For AX4 systems only
— Use the -upload switch to upload the HA report to the storage system only if you will use the Navisphere Software Assistant to install new software. We strongly recommend that you use the Software
Assistant, and not the -ndu install command to update your software. The
-ndu install
command does not perform the same pre-install validation checks as the Software Assistant. The Software Assistant is included in the Navisphere
Service Taskbar.
Checking the status of a server
To periodically check the high-availability status of a server, you can select option 3 from the server utility’s Welcome screen, and follow the instructions on the utility to generate a high-availability report, or issue the naviserverutilcli hav[-directory | -d outputdirectory] command.
where:
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
37
-directory | -d
outputdirectory
Specifies the directory to which you want to deposit the report on the server. This switch is required only if you are running the server utility from the CD. Otherwise the report is automatically saved to the
Navisphere Server Utility installation directory.
If the server utility is installed on your server and you specify a directory, the directory you specify will override the server utility’s installation directory.
Preparing for a software installation on a storage system
If you are generating a high-availability report as a prerequisite to updating storage-system software, you need to upload this report to the storage system prior to installing the software on the storage system.
To generate and upload an HA report, issue the following command from the directory in which you installed the utility:
naviserverutilcli hav -upload -ip IPaddress
|-h
IPaddress|network_name -user|-u username -password|-p
userpassword -scope|-s scopelevel -secfilepath securityfilepath
-port 443|2163
[-directory | -d outputdirectory] where:
38
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
-ip IPaddress
-h IPaddress |
network_name
-user | -u
username
-password | -p
userpassword
Specifies the IP address of an SP on the storage system to which you will upload the report. You cannot use this switch in conjunction with the -h switch.
Specifies the IP address or the network name of an SP on the storage system to which you will upload the report. You cannot use this switch in conjunction with the -ip switch.
Specifies the login name for the storage system. You must use this switch in conjunction with the -password | -p and -scope | -s switches. You
cannot use this switch in conjunction with the -secfilepath switch.
Specifies the password required for access to the storage system. You must use this switch in conjunction with the -user | -u and -scope | -s switches. You cannot use this switch in conjunction with the -secfilepath switch.
-scope | -s
scopelevel
-secfilepath
securityfilepath
Specifies the level of access this username and password has to the storage system: 0 = global; 1 = local; 2 = LDAP. You must use this switch in conjunction with the -user | -u and -password | -p switches. You
cannot use this switch in conjunction with the -secfilepath switch.
Searches the security file for the specified IP address. If the utility does not find the specified IP address, then the utility will use the default entry in the security file.
A security file is required to run this switch. For information on how to create a security file, refer to the EMC Navisphere Command Line (CLI)
Reference guide.
You cannot use this switch in conjunction with the -user | -u, -password |
-p , and -scope | -s switches.
-port 443 | 2163 Specifies the Navisphere Management Server port number. The default is 443.
-directory | -d
outputdirectory
Specifies the directory to which you want to deposit the report on the server. This switch is required only if you are running the server utility from the CD. Otherwise the report is automatically saved to the
Navisphere Server Utility installation directory.
If the server utility is installed on your server and you specify a directory, the directory you specify will override the server utility’s installation directory.
After uploading the HA report
In addition to running the server utility to validate server high availability, we strongly recommend that you perform the following manual checks.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
39
Verify support of software and hardware — Make sure that you have verified that all software and hardware is supported according to the EMC E-Lab™ Interoperability Navigator and support matrices.
This tool does not verify that you are running supported switch firmware, nor that you are using redundant switches (recommended).
Upgrading storage–system software — If you are about to perform a storage-system online software upgrade or you added an HBA, a
LUN, or changed connectivity (added a path or changed zoning): z Make sure that you have validated all software and hardware that are supported and interoperable according to EMC E-Lab
Interoperability Navigator and support matrices.
z Verify that all LUNs (virtual disks) that your servers will access during the upgrade are under path management software control. For servers running PowerPath and DMP, see the
Failover Software section on the Details tab of the report and locate these LUNs. If they are not in the list, the server will lose access to those LUNs during the software upgrade.
40
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Installing and removing the Admsnap Utility
EMC strongly recommends that you install admsnap on all servers connected to a storage system that has the SnapView driver installed.
Important:
If you are upgrading your storage system to FLARE OE version 02.19.xxx.5.yyy or later, you must upgrade admsnap to version
2.9.x or later. Before you upgrade FLARE, verify that your operating system version supports admsnap version 2.9.x or later. Refer to the
Admsnap Release Notes.
Installation prerequisites
Before you can install and use the Admsnap Utility, you must:
Install SnapView on a supported CLARiiON storage system. For a list of supported storage systems, refer to the release notes for
SnapView and admsnap.
Enable data access control on the same CLARiiON storage system.
Installing the Admsnap Utility on an AIX server
EMC recommends that you install the most recent version of the
Admsnap Utility software that is appropriate for your configuration.
You can download the most recent version from the software download page on the Powerlink website. You can also install the software from the server support CD; however, the CD may not contain the most recent version for your configuration.
1. On the AIX server, log in as superuser (root).
2. Open an xterm window and enter the following command to list currently installed admsnap software:
lslpp - L admsnap
3. Examine the list to determine if any previous revision of admsnap is present. If a previous revision is present, remove it as described
in Removing the Admsnap Utility on an AIX server, page 43.
4. To download the software for CX4 series, CX3 series, CX series, or
AX4–5 series storage systems with Navisphere Manager, do the following:
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
41
42
a. From the Navigator drop-down menu at the top right of the
Powerlink
website, select Software Downloads and Patches.
b. Select the appropriate Admsnap Utility version and download the zip file to a Windows host.
c. Unzip the zip file you downloaded.
d. Transfer the .lpp file to the usr/sys/inst.images folder on an AIX server.
5. To install the software from the server support CD, do the following: a. Insert the server support CD into the server’s CD drive.
b. Enter the following command at the AIX shell prompt:
crfs -v cdrfs -p ro -d cd0 -m /cd_tmp
c. Type the following commands to mount the CD, go to the
CD directory, copy the admsnap files to the correct hard disk directory, and recreate the table of contents:
mount /cd_tmp cd cd_tmp/AIX/admsnap/AIX5.x
cp * /usr/sys/inst.images
cd /usr/sys/inst.images
rm .toc
inutoc
6. If you are not in the SMIT (System Management Interface Tool)
System Management
window, open this window by entering smit
7. From the SMIT System Management window, follow this path:
Software Installation and Maintenance > Install and Update
Software > Install and Update from ALL Available Software
The Install and Update from LATEST Available Software window opens.
8. In this window, click List.
9. From the displayed list select the usr/sys/inst.images directory.
10. In the Install and Update from ALL Available Software window, click OK.
11. Click the List button for
SOFTWARE to Install
.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
12. From the list that appears, select the software that you want to install (admsnap) and click OK.
13. Click OK in the LATEST Available Software window.
14. When the confirmation window appears, click OK.
The installation process begins and the progress of the installation appears in the Install and Update from
ALL Available Software
window. The installation is complete when you see a message similar to the following:
Name Level Part Event Result admsnap XXXXX USR INSTALL Success
The software is installed in the directory /usr/admsnap.
Your installation is complete.
Removing the Admsnap Utility on an AIX server
1. At the root directory, start SMIT by entering:
smit
The SMIT System Management window opens.
2. From the SMIT System Management window, follow this path:
Software Installation and Maintenance > Software Maintenance and Utilities > Remove Installed Software
The Remove Installed Software window opens.
3. In the Remove Installed Software window, click the List button for
Software name
.
4. From the list that appears, select ADMSNAP, and click OK.
5. Click the List button for PREVIEW only? (remove operation will
NOT occur).
6. From the list that appears, click no.
7. Next, click the List button for
REMOVE dependent software
.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
43
8. From the list that appears, click yes.
9. Then click OK.
10. When the confirmation window appears, click OK.
The files are removed. The removal process is complete when you see a message similar to the following displayed in the bottom of the window:
Name Level Part Event Result admsnap XXXXX USR DEINSTALL Success
44
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
Running the Admsnap Utility
Refer to the EMC SnapView Command Line Interfaces Reference for information on running admsnap commands.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
45
46
Copyright© 2004–2008 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO
THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date regulatory document for your product line, go to the Technical
Documentation and Advisories section on EMC Powerlink.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on
EMC.com
All other trademarks mentioned herein are the property of their respective owners.
EMC CLARiiON Server Support Products for AIX Servers Installation Guide
ಡೌನ್ಲೋಡ್ ಮಾಡಿ
ಸಾರ್ವಜನಿಕ ಲಿಂಕ್ ನವೀಕರಿಸಲಾಗಿದೆ
ನಿಮ್ಮ ಚಾಟ್ಗೆ ಸಾರ್ವಜನಿಕ ಲಿಂಕ್ ಅನ್ನು ನವೀಕರಿಸಲಾಗಿದೆ. You may view and manage your chat links after registration here.
ದಾಖಲೆಯ ಸಾರ್ವಜನಿಕ ಲಿಂಕ್
ಜಾಹಿರಾತು