Sun Microsystems CP3260 User manual

Sun Netra™ CP3260 Blade Server
User’s Guide
Sun Microsystems, Inc.
www.sun.com
Part No. 820-0457-11
April 2009, Revision 01
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
This distribution may include materials developed by third parties.
Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in
the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd.
Sun, Sun Microsystems, the Sun logo, Netra, Sun Ray, the Netra logo and the Solaris logo are trademarks or registered trademarks of Sun
Microsystems, Inc., and its subsidiaries, in the U.S. and other countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other
countries. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems, Inc.
Use of any spare or replacement CPUs is limited to repair or one-for-one replacement of CPUs in products exported in compliance with U.S.
export laws. Use of CPUs as product upgrades unless authorized by the U.S. Government is strictly prohibited.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.
Cette distribution peut comprendre des composants développés par des tierces parties.
Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque
déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.
Sun, Sun Microsystems, le logo Sun, Netra, Sun Ray, le logo Netra et le logo Solaris sont des marques de fabrique ou des marques déposées de
Sun Microsystems, Inc. ou ses filiales, aux Etats-Unis et dans d’autres pays.
Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc.
aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun
Microsystems, Inc.
see above L’utilisation de pieces detachees ou d’unites centrales de remplacement est limitee aux reparations ou a l’echange standard d’unites
centrales pour les produits exportes, conformement a la legislation americaine en matiere d’exportation. Sauf autorisation par les autorites des
Etats-Unis, l’utilisation d’unites centrales pour proceder a des mises a jour de produits est rigoureusement interdite.
LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFACON.
Please
Recycle
Contents
Preface
1.
xiii
Introduction
1.1
Overview
1.2
Features
1.3
System Configurations
1.4
Advanced Rear Transition Modules
1.5
Hot-Swap Support
1.6
System Requirements and Options
1.7
1–1
1–2
1–8
1–11
1–14
1–14
1.6.1
Hardware Requirements and Options
1.6.2
Software Requirements
1–15
Part Number, Serial Number, and Revision Number
Identification 1–16
Hardware Installation
2–1
2.1
Equipment and Operator Safety
2.2
Materials and Tools Required
2–3
2.3
Preparing for the Installation
2–3
2.3.1
1–14
1–15
Technical Support and Warranty
1.7.1
2.
1–1
2–1
Checking Power, Thermal, Environmental, and Space
Requirements 2–3
iii
2.4
Installation Procedure Summary
2.5
Configuring On-Board Hardware
2.5.1
3.
4.
iv
2–4
2–5
Adding or Replacing FB-DIMM Memory Modules
2.5.1.1
To Remove FB-DIMM Memory Modules
2.5.1.2
To Install FB-DIMM Memory Modules
2.5.2
Adding or Replacing Compact Flash Card
2.5.3
Adding or Replacing TOD Clock Battery
2–10
2–11
2–13
2–14
2.6
Installing an Advanced Rear Transition Module (Optional)
2.7
Installing the Netra CP3260 Blade Server
2.8
Connecting External I/O Cables
2.9
Hot-Swapping the Netra CP3260 Blade Server
Software Installation
2–5
2–18
2–21
2–23
2–25
3–1
3.1
Operating Systems and Patches
3–1
3.2
Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3
2
3.3
Installing as a Diskless Clients
3–4
3.3.1
Creating a Boot Server for Diskless Clients
3.3.2
Adding a Diskless Client
3.3.3
Determining Local Network IP Addresses and Host Names
3–4
3–5
3.4
Firmware Updates
3–8
3.5
Configuring for 1-GbE or 10-GbE Switches
3.6
Downloading and Installing SunVTS Software
3–9
3.7
Formatting the Optional Compact Flash Card
3–10
Firmware and Blade Server Management
4.1
System Firmware
4.2
Power-On Self-Test Diagnostics
3–8
4–1
4–2
4–3
4.2.1
POST Test Coverage
4.2.2
POST Diagnostic and Error Message Format
Netra CP3260 Blade Server User’s Guide • April 2009
3–
4–3
4–4
3–7
4.3
OpenBoot Firmware
4.3.1
Getting to the ok Prompt
4.3.2
Auto-Boot Options
4.3.3
OpenBoot Commands
4.3.4
4–6
4–6
probe-scsi and probe-scsi-all Commands
4.3.3.2
probe-ide Command
4–8
4.3.3.3
show-devs Command
4–8
4.3.3.4
Using watch-net and watch-net-all Commands to
Check the Network 4–11
OpenBoot Configuration Variables
Error Handling Summary
4.5
Automatic System Recovery
4–16
Enabling and Disabling Automatic System Recovery
4.5.1.1
To Enable Automatic System Recovery
4.5.1.2
To Disable Automatic System Recovery
Network Device Aliases
4.7
Retrieving Device Information
4–19
4.8
Multiplexing to Zones 2 and 3
4–27
Hardware Architecture
4–17
4–17
4–17
4–18
Hardware Functional Descriptions
5.2
4–12
4–15
4.6
5.1
4–7
Viewing and Setting OpenBoot Configuration
Variables 4–14
4.4
4.5.1
4–5
4.3.3.1
4.3.4.1
5.
4–4
5–1
5–1
5.1.1
UltraSPARC T2 Processor and Memory
5.1.2
I/O
5.1.3
Base and Fabric Interfaces
5.1.4
Additional I/O
5.1.5
Power PC
5.1.6
ARTM Support
5–2
5–3
Hardware Modules
5–3
5–3
5–3
5–4
5–4
Contents
v
5.2.1
5.2.2
5.2.3
5.2.4
5.2.5
UltraSPARC T2 Processor
5–4
5.2.1.1
Electronic Fuse
5–7
5.2.1.2
Cores
5.2.1.3
L2 Cache
5.2.1.4
Memory Controller
5.2.1.5
I/O Interface
Memory Subsystem
5–8
5–9
Memory Capacity
5.2.2.2
Memory Speed
I/O Subsystem
5–9
5–9
5–9
5.2.3.1
PCI Express Switch
5.2.3.2
Base Interface
5.2.3.3
Fabric Interface
5.2.3.4
Common ARTM
Other ARTM Interfaces
5–10
5–10
5–10
5–10
5–11
5.2.4.1
Serial Ports
5.2.4.2
Ethernet Management Port
Front Panel I/O
5–11
5–11
5–11
5.2.5.1
Ethernet Management Port
5.2.5.2
Serial Port
5.2.5.3
Dual USB Ports
5–11
5–11
5.2.6
Compact Flash Socket
5.2.7
Service Processor MPC885
5.2.8
5–8
5–8
5.2.2.1
5.2.7.1
vi
5–8
5–12
5–12
5–12
Field-Programmable Gate Array
Intelligent Platform Management Controller
5–12
5–13
5.2.8.1
Intelligent Platform Management Bus
5.2.8.2
Interface to the PPC
5.2.8.3
IPMB-L Interface
Netra CP3260 Blade Server User’s Guide • April 2009
5–14
5–14
5–14
5.2.8.4
ATCA Hot-Swap Latch
5.2.8.5
LEDs
5.2.8.6
Power Control
5.2.8.7
System Monitor (ADM1026) and Thresholds
5.2.8.8
FRUID PROMs
5–15
5.2.10
ATCA Power Module (−48V to 12V)
5.2.11
TOD Clock Battery
Form Factor
A.2
Layout
A.3
Front Panel
5–17
5–18
5–18
A–1
A–1
A–1
A–3
A.3.1
Visual Indicators
A.3.2
Ports
A–3
A–3
Connectors and Pinout
A.4.1
5–15
5–17
I/O Subsystem Resets
A.1
A.4
5–15
5.2.9
A. Physical Characteristics
5–14
A–3
Front Panel Connectors
A–3
A.4.1.1
Ethernet Port
A–4
A.4.1.2
Dual USB Port
A.4.1.3
Serial Port
A–4
A–5
A.4.2
Compact Flash Connector
A.4.3
Midplane Power Connector (Zone 1)
A.4.4
Data Transport Connector (Zone 2)
A.4.5
Sun Netra ARTM Connectors (Zone 3)
A.4.6
TOD Clock Battery Holder
B. Sun OEM IPMI Commands
A–6
A–6
A–8
A–10
A–13
B–1
B.1
Get Version Command
B–2
B.2
Get RTM Status Command
B–3
Contents
vii
B.3
Solaris OS Graceful Shutdown Commands
B.4
Send Sensor State Command
Index
viii
Index–1
Netra CP3260 Blade Server User’s Guide • April 2009
B–5
B–4
Figures
FIGURE 1-1
Netra CP3260 Blade Server Front Panel
1–4
FIGURE 1-2
Netra CP3260 Blade Server (Top View)
1–5
FIGURE 1-3
Netra CP3260 Blade Server in an ATCA Shelf Enclosure
FIGURE 1-4
Netra CP3260 Blade Server, Midplane, and Netra CP32x0 ARTM
FIGURE 1-5
Netra CP3260 Blade Server Barcode Labeling
FIGURE 2-1
FB-DIMM Memory Locations
FIGURE 2-2
Removing an FB-DIMM Memory Module
FIGURE 2-3
Installing a FB-DIMM Memory Module
FIGURE 2-4
Compact Flash Card Location
FIGURE 2-5
TOD Battery Location
FIGURE 2-6
Installing a Netra CP32X0 ARTM
FIGURE 2-7
Installing Blade Server Into Chassis Slot
FIGURE 2-8
Netra CP3260 Blade Server Latches and Locking Screws
FIGURE 2-9
Hot-Swap Latch and Hot-Swap LED
FIGURE 5-1
Netra CP3260 Blade Server Block Diagram
FIGURE 5-2
UltraSPARC T2 Multicore Processor Block Diagram
FIGURE A-1
Netra CP3260 Blade Server Layout
FIGURE A-2
Ethernet RJ-45 Connector
FIGURE A-3
Dual USB Connector
FIGURE A-4
Front Panel Serial Port Diagram
1–9
1–12
1–17
2–6
2–11
2–12
2–14
2–16
2–19
2–22
2–23
2–26
5–2
5–5
A–2
A–4
A–5
A–6
ix
FIGURE A-5
Power Distribution Connector (Zone 1) P10
FIGURE A-6
Zone 2 Connectors
FIGURE A-7
Zone 3 Signal Connectors
FIGURE A-8
Zone 3 Power Connector
FIGURE A-9
TOD Battery Location
x
A–9
A–10
A–12
A–14
Netra CP3260 Blade Server User’s Guide • April 2009
A–7
Tables
TABLE 1-1
Netra CP3260 Blade Server Feature Summary
1–2
TABLE 1-2
I/O Configurations
TABLE 2-1
Ethernet Device Names
TABLE 3-1
Local Network Information
TABLE 4-1
Ways of Accessing the ok Prompt
4–5
TABLE 4-2
OpenBoot Configuration Variables
4–12
TABLE 4-3
Network Device Aliases
TABLE 4-4
PICL Frutree Entries and Description for the Netra CP3260 Board
TABLE 5-1
Voltage Sensor Thresholds
TABLE 5-2
CPU Temperature Alarms
TABLE A-1
Ethernet Port Connector Pin Assignments
TABLE A-2
USB Connector Pin Assignments
TABLE A-3
Serial Port RJ-45 Connector Pinouts
TABLE A-4
Power Distribution Connector Pin Assignments
TABLE A-5
Zone 2 J23 Connector Pin Assignments
A–9
TABLE A-6
Zone 2 J20 Connector Pin Assignments
A–9
TABLE A-7
Zone 3 J31 Connector Pin Assignments
A–11
TABLE A-8
Zone 3 J32 Connector Pin Assignments
A–11
TABLE A-9
Zone 3 J33 Connector Pin Assignments
A–12
TABLE A-10
Zone 3 Power Connector Pin Assignments
1–13
2–24
3–7
4–18
4–19
5–16
5–16
A–4
A–5
A–6
A–7
A–13
xi
TABLE B-1
Get Version Command Data Bytes
TABLE B-2
Get RTM Status Command Data Bytes
TABLE 1
Solaris OS Graceful Shutdown Parameters in /etc/fsmd.conf
TABLE B-3
Send Sensor State Command Data Bytes
xii
Netra CP3260 Blade Server User’s Guide • April 2009
B–2
B–3
B–5
B–4
Preface
The Sun Netra CP3260 Blade Server User’s Guide provides information about features,
installation, configuration, functional hardware components, and physical properties
of this blade server. The Sun Netra CP3260 Blade Server User’s Guide is written for
system integration engineers, field applications and service engineers, and others
involved in the integration of these blade servers into systems.
How This Document Is Organized
Chapter 1 provides an overview of the Sun Netra™ CP3260 blade server.
Chapter 2 provides instructions on hardware installation.
Chapter 3 provides instructions on the software configuration.
Chapter 4 provides information about the Netra CP3260 firmware.
Chapter 5 provides hardware and functional descriptions of the Netra CP3260 blade
server.
Appendix A provides information about the physical characteristics of the Netra
CP3260 blade server.
Appendix B describes the Sun-specific OEM-defined Intelligent Platform
Management Interface (IPMI) commands.
xiii
Using UNIX Commands
This document might not contain information on basic UNIX® commands and
procedures such as shutting down the system, booting the system, and configuring
devices. Refer to the following for this information:
■
Software documentation that you received with your system
■
Solaris™ Operating System documentation, which is at:
http://docs.sun.com/app/docs/prod/solaris
Shell Prompts
xiv
Shell
Prompt
C shell
machine-name%
C shell superuser
machine-name#
Bourne shell and Korn shell
$
Bourne shell and Korn shell superuser
#
Sun Netra CP3260 Blade Server User’s Guide • April 2009
Typographic Conventions
Typeface
Meaning
Examples
AaBbCc123
The names of commands, files,
and directories; on-screen
computer output
Edit your.login file.
Use ls -a to list all files.
% You have mail.
AaBbCc123
What you type, when contrasted
with on-screen computer output
% su
Password:
AaBbCc123
Book titles, new words or terms,
words to be emphasized.
Replace command-line variables
with real names or values.
Read Chapter 6 in the User’s Guide.
These are called class options.
You must be superuser to do this.
To delete a file, type rm filename.
Note – Characters display differently depending on browser settings. If characters
do not display correctly, change the character encoding in your browser to Unicode
UTF-8.
Related Documentation
The Netra CP3260 blade server documentation is listed in the following table. Except
for the Important Safety Information for Sun Hardware Systems, all the documents listed
are available online at:
http://docs.sun.com/app/docs/prod/cp3260.brd
Title
Part Number
Netra CP3260 Board Product Notes
820-0455
Netra CP3260 Board Getting Started Guide
820-0456
Important Safety Information for Sun Hardware Systems (printed version only)
816-7190
Preface
xv
Application
Title
Part Number
Format
Location
Installation and
Configuration
Sun Netra CP32x0 SAS Storage
Advanced Rear Transition Module,
Dual HD User’s Guide
820-3147
PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.sas#hic
Installation and
Configuration
Sun Netra™ CP32x0 Quad GbE,
Dual Fibre Channel, Advanced Rear
Transition Module, User’s Guide
820-3148
PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.4gbefc?l=en#hic
Installation and
Configuration
Sun Netra™ CP32x0 10GbE
Advanced Rear Transition Module,
Dual Port User’s Guide
820-3150
PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.10gbee?l=en#hic
Documentation, Support, and Training
Sun Function
URL
Documentation
http://www.sun.com/documentation/
Support
http://www.sun.com/support/
Training
http://www.sun.com/training/
Third-Party Web Sites
Sun is not responsible for the availability of third-party web sites mentioned in this
document. Sun does not endorse and is not responsible or liable for any content,
advertising, products, or other materials that are available on or through such sites
xvi
Sun Netra CP3260 Blade Server User’s Guide • April 2009
or resources. Sun will not be responsible or liable for any actual or alleged damage
or loss caused by or in connection with the use of or reliance on any such content,
goods, or services that are available on or through such sites or resources.
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun Netra CP3260 Blade Server User’s Guide, part number 820-0457-11
Preface
xvii
xviii
Sun Netra CP3260 Blade Server User’s Guide • April 2009
1-C HR
APTE
CHAPTER
Introduction
This chapter contains the following sections:
1.1
■
Section 1.1, “Overview” on page 1-1
■
Section 1.2, “Features” on page 1-2
■
Section 1.3, “System Configurations” on page 1-8
■
Section 1.5, “Hot-Swap Support” on page 1-14
■
Section 1.6, “System Requirements and Options” on page 1-14
■
Section 1.7, “Technical Support and Warranty” on page 1-15
Overview
The Netra CP3260 blade server (FIGURE 1-1 and FIGURE 1-2) is a high-performance
single-board computer based on one UltraSPARC® T2 multicore processor and
designed for high availability in a switched network computing environment. This
blade server is compliant with Advanced Telecommunications Computing
Architecture (AdvancedTCA®, ATCA®) specifications (PICMG® 3.0 and PICMG 3.1).
The PICMG (PCI Industrial Computer Manufacturers Group) standards committee
has developed the new ATCA (or PICMG 3.x) standard to address the issues posed
by previous standards based on cPCI and cPSB (PICMG 2.x). The PICMG 3.x
specification brought the following changes to the existing PICMG 2.x family of
products:
■
■
Larger board space (8U high compared to 6U for cPCI), which allows more
features and processing power
On-board power supplies deriving local power from redundant −48V power from
the midplane (rather than separate power supplies)
■
6-HP slot width, allowing greater component height
■
Advanced Mezzanine card (AMC) support and options
1-1
■
Elimination of PCI connectivity between the blade servers in the system and
reallocation of connectivity to serial interconnects, eliminating single points of
failure
■
Mandatory use of Intelligent Platform Management Interface (IPMI) management
interfaces
■
Flexible user I/O
■
Power and thermal management guidelines enforced by the management
infrastructure
■
Separation of control and data traffic by supporting the Base (PICMG 3.0) and
Fabric (PICMG 3.1) interfaces
The ATCA standard consists of the PICMG 3.0, PICMG 3.1, PICMG 3.2, and PICMG
3.3 specifications. The Netra CP3260 blade server complies with:
1.2
■
PICMG 3.0, the base specification that defines the mechanical, power distribution,
system management, data transport, and regulatory guidelines
■
PICMG 3.1, which builds upon the PICMG 3.0 base specification and the IEEE
802.3-2003 standard
Features
TABLE 1-1 provides a summary of Netra CP3260 features.
TABLE 1-1
1-2
Netra CP3260 Blade Server Feature Summary
Feature
Description
CPU
• One UltraSPARC T2 processor (6 or 8 cores, 8 threads/core)
• CPU core speed: 1.2 GHz
Memory
• Fully-Buffered DDR2-based memory DIMMs (FB-DIMMs)
• Eight FB-DIMM slots (one per channel or two per memory controller)
• 1-Gbyte or 2-Gbyte FB-DIMMs, for a total memory of 16 Gbytes
Service
processor
• Service processor (MPC885) for CPU reset, boot, partition, and fault
management architecture (FMA)
Power
requirement
• 200 watts maximum
• Dual redundant input 48V to derive on-board power
Cooling
requirement
• 32 - 35 cfm at 55C
Netra CP3260 Blade Server User’s Guide • April 2009
TABLE 1-1
Netra CP3260 Blade Server Feature Summary
Feature
Description
PICMG
compliance
•
•
•
•
•
•
Node board
support
Functions as a CPU node board with the Solaris operating system and
software packages
Operating
system
Solaris 10 8/07 OS and subsequent compatible versions with supported
Netra patches
Internal I/O
(connections to
ATCA
midplane)
• Dual 10/100/1000BASE-T Ethernet for Base interfaces
• 10-Gb XAUI or 1-Gb SERDES Ethernet interfaces as Fabric interfaces
• Dual IPMI channel connects to the midplane for communicating with
the Shelf Management card
External I/O
•
•
•
•
IPMI system
management
Uses IPMI communications with baseboard management controller
(BMC); performs advanced system monitoring (ASM) on local board
interface (for example, temperature sense, FRU ID, and control)
Hot-swap
support
• Board hot-swap support.
Front panel
access I/O
• One 10/100/1000BASE-T Ethernet port (RJ-45)
• Dual USB 2.0 compliant ports
• One serial port (RJ-45)
Rear access
interface
• Common rear transition module (RTM) interface supporting the Sun
Netra CP32X0 ARTMs
Compact Flash
support
• Compact Flash socket to support a Sun 8-Gbyte or 16-Gbyte user flash
type I/II memory card
Building
compliance
Network Equipment Building Systems (NEBS) Level 3
Flash update
Supported from downloaded file
Single-wide ATCA module
ATCA 8U form factor
Single-wide ATCA module
6-HP slot width
PICMG 3.0 R1.0
PICMG 3.1 R1.0
One 10/100/1000BASE-T Ethernet maintenance port on front panel
One asynchronous serial port for console on front panel
Dual USB 2.0 compliant ports on the front panel
Rear access available on Netra CP32x0 ARTMs:
– One 10/100/1000BASE-T Ethernet port (RJ-45)
– One asynchronous serial port (RJ-45)
Note - With a Netra CP32x0 ARTM installed, both the front panel and
ARTM serial ports are active. Either serial port can be used; but only one
of the serial ports can be used at a time.
Chapter 1
Introduction
1-3
FIGURE 1-1
1-4
Netra CP3260 Blade Server Front Panel
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 1-2
Netra CP3260 Blade Server (Top View)
Chapter 1
Introduction
1-5
⑤
④
③
②
①
⑦
1-6
Netra CP3260 Blade Server User’s Guide • April 2009
Figure Legend
1
Top latch
2
Out of Service LED (yellow)
3
Ethernet management port (RJ-45)
4
OK LED (green)
5
Dual USB Ports (USB 2.0)
6
Serial (console) port (RJ-45, ttya)
7
Hot-Swap LED (blue)
8
Hot-Swap switch and bottom latch
Figure Legend
1
Zone 1 power connector
2
Zone 2 signal connectors
3
Zone 3 RTM signal connectors
4
Zone 3 ARTM power connector
Chapter 1
Introduction
1-7
Figure Legend
1.3
5
Compact Flash connector
6
UltraSPARC T2 processor (under heat sink)
7
FB-DIMMs
System Configurations
Netra CP3260 blade servers can be installed in an ATCA shelf (or chassis), as shown
in FIGURE 1-3. The blade servers can be deployed in various electrical configurations
to suit each end-user requirement. For example, the blade server can be configured
to boot from a network as a diskless client with either a front panel or RTM network
connection, or from an optional Compact Flash card. Also, a Netra CP32x0 SAS
Storage ARTM, Dual HD can be installed to provide local disk I/O, which can be
used optionally as a boot path.
1-8
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 1-3
Netra CP3260 Blade Server in an ATCA Shelf Enclosure
Chapter 1
Introduction
1-9
①
②
③
④
Ethernet
⑤
⑥
⑦
Diskless client that boots through network from a remote server
1-10
Netra CP3260 Blade Server User’s Guide • April 2009
Figure Legend
1.4
1
Remote server
2
Sun Netra CP32X0 Advanced RTM (installed from rear)
3
Serial connection
4
Console terminal
5
Ethernet connection (RJ-45)
6
Sun Netra CP3260 blade server (installed from front)
7
Netra CT 900 server ATCA shelf
Advanced Rear Transition Modules
An optional Sun Netra CP32X0 Advanced Rear Transition Module (ARTM) can be
installed into the rear of the ATCA enclosure, opposite the Netra CP3260 blade
server (FIGURE 1-3). The Netra CP32x0 ARTM connects to the Netra CP3260 blade
server’s Zone 3 rear I/O connectors (FIGURE 1-2). Contact your Sun representative for
information on the Netra CP32x0 ARTMs.
FIGURE 1-4 shows the physical relationship between the blade server, the rear
transition module, and the midplane in a typical ATCA system.
Chapter 1
Introduction
1-11
FIGURE 1-4
Netra CP3260 Blade Server, Midplane, and Netra CP32x0 ARTM
①
②
③
1-12
Netra CP3260 Blade Server User’s Guide • April 2009
Figure Legend
1
Sun Netra CP3260 blade server
2
Sun Netra CP32X0 ARTM
3
ATCA chassis midplane
Note – When a Netra CP32x0 ARTM is used with the Netra CP3260 blade server,
shielded cables are required for serial I/O ports. Unshielded cables can be used on
Ethernet ports to satisfy EMI compliance standards. The shields for all shielded
cables must be terminated on both ends.
The customer can order a Netra CP32x0 ARTM, build a custom card, or buy from an
independent hardware vendor (IHV). A minimal set of I/O must provide a boot
path for the host blade server and a path for console I/O to deliver commands and
to read blade server and system status.
Possible boot and console configurations are described in TABLE 1-2. Sun
Microsystems supplies the Netra CP3260 blade server and compatible Netra CP32x0
ARTMs. The other configurations require (IHV) hardware.
TABLE 1-2
I/O Configurations
I/O
Hardware Required
Description
Ethernet
Netra CP3260 blade server
Default boot path uses the Ethernet port; when the blade server
runs in diskless client configuration.
Netra CP32x0 ARTM
(ARTM—supplied as an option
for rear access)
SAS
Netra CP32x0 SAS Storage
ARTM, Dual HD
SAS devices can be used for local booting.
Serial data
Netra CP3260 blade server
Netra CP32x0 ARTMs
Serial port on front panel provides a path for a console I/O.
The serial port can be used on either the Netra CP3260 blade
server or Netra CP32x0 ARTM, but both ports cannot be used at
the same time. If the serial port on Netra CP3260 blade server is
used, the serial port on the Netra CP32x0 ARTMs should not be
used and vise versa.
Compact
Flash
Sun Compact Flash card
The Compact Flash connector can be used to add an optional
Sun 8-Gbyte or 16-Gbyte user flash type I/II memory card.
Chapter 1
Introduction
1-13
1.5
Hot-Swap Support
There are three hot-swap models described in the PICMG ATCA specification: basic
hot-swap, full hot-swap, and high-availability (HA) hot-swap. Refer to the PICMG
ATCA Specification, which provides a detailed description of this subject.
The Netra CP3260 blade server supports basic hot-swap. The hot-swap process uses
hardware connection control to disconnect and connect the hardware in an orderly
sequence.
1.6
System Requirements and Options
This section contains the system-level hardware and software requirements and
options for the Netra CP3260 blade server.
1.6.1
Hardware Requirements and Options
Sun provides the following items for customer order:
■
Netra CP3260 blade server
■
The Netra CT 900 server fan tray upgrade kit (part number: 594-4953)
This upgrade kit must be installed in the Netra CT 900 server chassis to provide
adequate cooling. Refer to the Netra CT 900 Server Upgrade Guide (820-3255) for
more information. (Also see Section 2.7, “Installing the Netra CP3260 Blade
Server” on page 2-21 for fan tray upgrade verification information.)
■
(Optional) Netra CP32x0 ARTMs
An RTM enables rear system I/O access to the network, to a boot device, or to a
console terminal. The Netra CP32x0 ARTMs also provide a variety of I/O and
storage solutions, an AMC-like interface, and hot-swap capability.
The Netra CP32x0 ARTMs are optional and must be ordered separately. Contact
your Sun representative for information on the Netra CP32x0 ARTMs and
compatibility with the Netra ATCA node boards.
■
(Optional) Compact Flash card
An IDE Compact Flash card is optional and must be ordered separately.
Acquire the following components, if needed:
1-14
Netra CP3260 Blade Server User’s Guide • April 2009
■
Serial terminal or terminal emulation for console output.
■
Cables for terminal and network connections.
■
Optional AMC disks and PCI-E cards.
The following are ATCA and other minimum requirements met by the Netra CP3260
blade server.
1.6.2
■
ATCA system enclosure for 8U boards (includes chassis, midplane, power supply)
■
Console output device or serial terminal
■
Boot device (such as hard drive, network, or Compact Flash card
■
Peripheral device for network access
■
Intelligent Platform Management Controller (IPMC)
Software Requirements
The Netra CP3260 blade server supports Solaris 10 8/07 OS and subsequent
compatible versions
Refer to the Netra CP3260 Blade Server Product Notes (820-0455) for more Solaris OS
information, including a list of the required software patches. You can view and
download the latest version of this manual at the following web site:
http://docs.sun.com/app/docs/prod/netra.brds
1.7
Technical Support and Warranty
Should you have any technical questions or support issues that are not addressed in
the Netra CP3260 blade server documentation set or on the web site, contact your
local Sun Services representative. This hardware carries a one-year return-to-depot
warranty. For customers in the US or Canada, call 1-800-USA-4SUN (1-800-872-4786).
For customers in the rest of the world, find the World Wide Solution Center nearest
you by visiting our web site:
http://www.sun.com/service/contacting/solution.html
When you call Sun Services, be sure to indicate if the Netra CP3260 blade server was
purchased separately and is not associated with a system. Have the proper blade
server identification information ready. Be prepared to give the representative the
blade server part number, serial number, and date code (FIGURE 1-5).
Chapter 1
Introduction
1-15
1.7.1
Part Number, Serial Number, and Revision
Number Identification
The Netra CP3260 blade server part number, serial number, and revision can be
found on labels located on the card (FIGURE 1-5). The Sun barcode labels provide the
following information:
■
SunSN – Sun serial number (for example, 1005LCB-0626WM001M)
■
SunPN – Sun part number and dash number (for example, 501-7658-01), -01 is the
dash number
■
Rev – Revision number of the part (for example: Rev 06)
The Media Access Control (MAC) address label contains the MAC address for the
blade server in printed and barcode form.
1-16
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 1-5
Netra CP3260 Blade Server Barcode Labeling
Note – You might find the labels shown in
FIGURE 1-5 on other locations on your
blade server. Your particular blade server configuration might also appear different
than the illustration.
Chapter 1
Introduction
1-17
1-18
Netra CP3260 Blade Server User’s Guide • April 2009
2-C HR
APTE
CHAPTER
Hardware Installation
This chapter describes the hardware installation procedures for the Netra CP3260
blade server, and contains the following sections:
2.1
■
Section 2.1, “Equipment and Operator Safety” on page 2-1
■
Section 2.2, “Materials and Tools Required” on page 2-3
■
Section 2.3, “Preparing for the Installation” on page 2-3
■
Section 2.4, “Installation Procedure Summary” on page 2-4
■
Section 2.5, “Configuring On-Board Hardware” on page 2-5
■
Section 2.6, “Installing an Advanced Rear Transition Module (Optional)” on
page 2-18
■
Section 2.8, “Connecting External I/O Cables” on page 2-23
Equipment and Operator Safety
Refer to Important Safety Information for Sun Hardware Systems (816-7190) for general
safety information.
Read the safety statements specific to the Netra CP3260 blade server carefully before
you install or remove any part of the system.
Caution – Depending on the particular chassis design, operations with open
equipment enclosures can expose the installer to hazardous voltages with a
consequent danger of electric shock. Ensure that line power to the equipment is
disconnected during operations that make high voltage conductors accessible.
2-1
The installer must be familiar with commonly accepted procedures for integrating
electronic systems and with the general practice of Sun systems integration and
administration. Although parts of these systems are designed for hot-swap
operation, other components must not be subjected to such stresses. Work with
power connected to a chassis only when necessary, and follow these installation
procedures to avoid equipment damage.
This equipment is sensitive to damage from electrostatic discharge (ESD) from
clothing and other materials. Use the following antistatic measures during an
installation:
■
If possible, disconnect line power from the equipment chassis when servicing a
system or installing a hardware upgrade. If the chassis cannot be placed upon a
grounded antistatic mat, connect a grounding strap between the facility electrical
input ground (usually connected to the equipment chassis) and facility electrical
service ground.
■
Use an antistatic wrist strap when:
■
Removing a blade server from its antistatic bag
■
Connecting or disconnecting blade servers or peripherals
The other end of the strap lead should be connected to one of the following:
2-2
■
A ground mat
■
Grounded chassis metalwork
■
A facility electrical service ground
■
Keep blade servers in the antistatic bags until they are needed.
■
Place circuit blade servers that are out of their antistatic bags on an antistatic mat
if one is available. The mat must be grounded to a facility electrical service
ground. Do not place blade servers on top of an antistatic bag unless the outside
of the bag also has antistatic protective properties.
■
Remove a blade server from its antistatic bag only when wearing a properly
connected ground strap.
Netra CP3260 Blade Server User’s Guide • April 2009
2.2
Materials and Tools Required
This section provides information on the materials and tools required to perform
installation. The minimum tools required to perform installation are:
■
Phillips screwdrivers, No. 1, No. 2 (optional)
■
Antistatic wrist strap
■
Terminal console
See Section 1.6.1, “Hardware Requirements and Options” on page 1-14 for
information on hardware requirements.
2.3
Preparing for the Installation
Prepare for installation by reading and performing the following steps:
1. Become familiar with the contents of the documentation referenced in the steps.
2. Verify that all listed hardware and software are available (see Section 1.6, “System
Requirements and Options” on page 1-14).
3. Check power, thermal, environmental, and space requirements (see Section 2.3.1,
“Checking Power, Thermal, Environmental, and Space Requirements” on
page 2-3).
4. Verify that local area network (LAN) preparations are completed (see
Section 3.3.3, “Determining Local Network IP Addresses and Host Names” on
page 3-7).
5. Ensure that the host names and their network IP addresses are allocated and
registered at the site.
2.3.1
Checking Power, Thermal, Environmental, and
Space Requirements
Verify that you meet the following requirements:
■
Your enclosure specifications support the sum of the specified maximum blade
server power loads.
Chapter 2
Hardware Installation
2-3
2.4
■
Facility power loading specifications can support the rack or enclosure
requirements.
■
Your enclosure specifications support the cooling airflow requirements. The Netra
CP3260 blade server fits a standard ATCA shelf or chassis. If your installation
requirements are different, contact your field application engineer.
Installation Procedure Summary
The procedure in this section summarizes the Netra CP3260 blade server installation
at a high level. Be sure to read the details in Section 2.5, “Configuring On-Board
Hardware” on page 2-5 before installing the blade server.
The procedure to set up and configure a Netra CP3260 blade server in a system
includes the following actions:
1. Configure the on-board physical hardware. For example, add memory or optional
Compact Flash.
2. Configure and install a rear transition module (RTM), if necessary.
3. Physically install the Netra CP3260 blade server into the chassis.
4. Connect the nodes to a local network. Alternatively, the blade server can be run as
a standalone system without a network connection.
5. Install the operating system and patches, as necessary. See Section 3.1, “Operating
Systems and Patches” on page 3-1.
6. Download firmware updates, if needed. See Section 3.4, “Firmware Updates” on
page 3-8.
7. Configure the operating system for 1-GbE or 10-GbE Ethernet switches. See
Section 3.5, “Configuring for 1-GbE or 10-GbE Switches” on page 3-8
2-4
Netra CP3260 Blade Server User’s Guide • April 2009
2.5
Configuring On-Board Hardware
This section provides the procedures for adding or replacing the on-board hardware
components such as memory modules, a Compact Flash card, and the time-of day
(TOD) battery. Read and perform the procedures, as necessary, before installing the
Netra CP3260 blade server into the chassis.
2.5.1
Adding or Replacing FB-DIMM Memory Modules
The Netra CP3260 blade server supports a total of 8 FB-DIMMs and a maximum
memory capacity of 16 Gbytes (using eight 2-Gbyte DIMMs). FIGURE 2-1 shows the
location of the FB-DIMMs. The Netra CP3260 blade server accommodates the
following:
■
Eight standard FB-DIMMs, buffered, and registered
■
1-Gbyte or 2-Gbyte FB-DIMM modules
Note – You cannot mix 1-Gbyte and 2-Gbyte FB-DIMMs. All eight DIMMs must be
the same density (that is, all 1-Gbyte or all 2- Gbyte FB-DIMMs).
The Netra CP3260 blade server supports FB-DIMM memory modules that have the
following characteristics:
■
Each DIMM has a 72-bit-wide data bus (64+8 ECC) and up to 14 address bits.
■
Memory controller supports 128-bit data plus 9-bit error-correcting code (ECC).
■
Maximum of 16 Gbytes (eight 2-Gbyte FB-DIMMs) or 8 Gbyte (eight 1-Gbyte
FB-DIMMs).
■
FB-DIMM @1.55 volts or [email protected] volts
For information about sensors, refer to the Netra CT 900 Software Developer’s Guide
(819-1175).
For additional information, see Section 5.2.2, “Memory Subsystem” on page 5-9.
FIGURE 2-1 shows the location of the DIMMs.
Chapter 2
Hardware Installation
2-5
FIGURE 2-1
2-6
FB-DIMM Memory Locations
Netra CP3260 Blade Server User’s Guide • April 2009
Chapter 2
Hardware Installation
2-7
2-8
Netra CP3260 Blade Server User’s Guide • April 2009
① ②③ ④
⑤ ⑥ ⑦ ⑧
Figure Legend
1
FB0B
2
FB0A
3
FB1B
4
FB1A
DIMM
Pair 0
5
FB2A
6
FB2B
DIMM
Pair 1
7
FB3A
8
FB3B
DIMM
Pair 2
DIMM
Pair 3
Chapter 2
Hardware Installation
2-9
2.5.1.1
To Remove FB-DIMM Memory Modules
You might need to remove a FB-DIMM module from the Netra CP3260 blade server
if you are returning the FB-DIMM module or the blade server for service, or if you
are replacing a module with another FB-DIMM module.
Note – Safely store the original factory-shipped FB-DIMM and related FB-DIMM
packaging. You might wish to store any removed FB-DIMM in the new FB-DIMM
packaging, or use the packaging for service.
To remove a FB-DIMM from the Netra CP3260 blade server, perform the following
steps:
1. Take antistatic precautions: attach and electrically ground the wrist strap.
Caution – Always wear a grounded antistatic wrist strap when handling modules.
2. Place the Netra CP3260 blade server on an antistatic mat, or on the blade
server’s antistatic bag if you do not have a mat available.
3. For the FB-DIMM you wish to remove, simultaneously pull both FB-DIMM
retainer clips outward from the slot (see ➊ in FIGURE 2-2).
4. Grasp the FB-DIMM by the edges, and carefully pull it out of its connector (see
➋ in FIGURE 2-2). A slight rocking motion might be required.
Caution – Take extra care to prevent damage when removing the outer most
FB-DIMMs.
5. Place it in an antistatic bag.
2-10
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 2-2
Removing an FB-DIMM Memory Module
6. If you are replacing the module you removed with a new FB-DIMM, install it
as described in Section 2.5.1.2, “To Install FB-DIMM Memory Modules” on
page 2-11.
2.5.1.2
To Install FB-DIMM Memory Modules
The following procedure provides a general guide for installing additional memory.
However, for directions on the installation process of the memory FB-DIMMs on the
Netra CP3260 blade server, refer to the documentation that shipped with the
memory module.
Caution – Do not remove the FB-DIMM from its antistatic container until you are
ready to install it on the card. Handle the module only by its edges. Do not touch
module components or metal parts. Always wear a grounded antistatic wrist strap
when handling modules.
Chapter 2
Hardware Installation
2-11
1. Locate the FB-DIMM connectors on the Netra CP3260 blade server.
Select the connectors where you will install the memory module (FIGURE 2-1). If
you need to replace an existing memory module with a new module, see
Section 2.5.1.1, “To Remove FB-DIMM Memory Modules” on page 2-10 for
instructions on removing the FB-DIMM module.
2. Remove the FB-DIMM from its protective packaging, holding the module only
by the edges.
3. Insert the bottom edge of the FB-DIMM into the bottom of the slot’s
hinge-style connector (see ➊ in FIGURE 2-3).
The socket and module are both keyed, which means the module can be installed
one way only. With even pressure, push simultaneously on both upper corners of
the FB-DIMM until its bottom edge (the edge with the gold fingers) is firmly
seated in the connector.
FIGURE 2-3
Installing a FB-DIMM Memory Module
Caution – Do not rock the FB-DIMM into place. Ensure that all contacts engage at
the same time. You will feel or hear a click when the FBDIMM properly seats in the
connector.
2-12
Netra CP3260 Blade Server User’s Guide • April 2009
4. Press the top edge of the FB-DIMM toward the blade server until the retainer
clips click into place (see ➋ in FIGURE 2-3).
The small retainer clips on each side of the FB-DIMM slot click into place in the
notches on the sides of the FB-DIMM.
2.5.2
Adding or Replacing Compact Flash Card
You can install an optional Sun Compact Flash card on the Netra CP3260 blade
server. The Compact Flash card is not hot-swappable and there is no access to the
card once the blade server is installed in an ATCA chassis.
To install the Compact Flash card, use the arrow on the card’s label as a guide and
insert the card into the Compact Flash connector (FIGURE 2-4).
Note – On the Netra CP3260, the Compact Flash is an USB removable media device.
Therefore, you must use the Solaris rmformat utility to format the device. Refer to
the rmformat(1) man page for more information.
Note – Sun Compact Flash cards have a life of 2,000,000 write/erase cycles. Users
are responsible for ensuring that the operating system and applications do not
exceed this limitation.
If the Solaris OS is installed on the Compact Flash card, Sun recommends that you
reconfigure the swap space to another storage device.
Chapter 2
Hardware Installation
2-13
FIGURE 2-4
Compact Flash Card Location
①
Figure Legend
1
2.5.3
Compact Flash connector
Adding or Replacing TOD Clock Battery
FIGURE 2-5 shows the location of the TOD clock battery and holder.
The optional TOD battery must be type CR1632, with a minimum of 4ma abnormal
charging current rating (for example; a Renata CR1632).
2-14
Netra CP3260 Blade Server User’s Guide • April 2009
Caution – Risk of explosion if the battery is replaced by an incorrect type.
Dispose of batteries properly in accordance with manufacturer’s instructions and
local regulations.
To install the battery, perform the following steps:
1. Remove the FB-DIMM closest to the front panel.
To do this, see Section 2.5.1.1, “To Remove FB-DIMM Memory Modules” on
page 2-10.
2. (Optional) Remove the old battery if necessary.
3. Slide the new battery into the holder with the side labeled “+ “ facing up.
4. Re-install the FB-DIMM.
To do this, see Section 2.5.1.2, “To Install FB-DIMM Memory Modules” on
page 2-11.
Chapter 2
Hardware Installation
2-15
FIGURE 2-5
2-16
TOD Battery Location
Netra CP3260 Blade Server User’s Guide • April 2009
①
②
③
Chapter 2
Hardware Installation
2-17
Figure Legend
2.6
1
TOD battery location
2
Remove this FB-DIMM for access to battery
3
TOD clock battery location with FB-DIMM removed
Installing an Advanced Rear Transition
Module (Optional)
For rear I/O access, a compatible RTM must be used with the Netra CP3260 blade
server.
Note – If you are using a Netra CP32x0 ARTM, refer to the appropriate Netra
CP32x0 ARTM User’s Guide for specific installation instructions.
In general, install the RTM from the rear of the server. Look at the front of the server
and locate the slot number where the Netra CP3260 blade server is installed. Then
go to the back of the server and install the RTM in that particular slot.
2-18
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 2-6
Installing a Netra CP32X0 ARTM
②
①
③
Figure Legend
1
Netra CP3260 blade server
2
Netra CP32X0 ARTM
3
ATCA chassis midplane
Chapter 2
Hardware Installation
2-19
2.7
Installing the Netra CP3260 Blade Server
Caution – The Netra CT 900 server fan tray upgrade kit (594-4953) must be
installed in the chassis before the Netra CP3260 blade server is installed. This fan
tray upgrade is required to provide adequate cooling and to prevent the system from
overheating or shutting down due to an over-temperature condition that can occur
with the older fan trays.
To verify that the fan tray upgrade is installed on a Netra CT 900 server, log into the
Shelf Manager and issue the following command for fan trays 1, 2, and 3 (or fan
trays 0, 1, and 2 for older chassis):
clia fruinfo fan_tray 1 | grep "Product Part"
If Product Part / Model Number = 371-3033-xx is displayed, an upgraded
fan tray is installed. If Product Part / Model Number is not 371-3033, an old fan
tray is installed and the upgrade is required. Refer to the Netra CT 900 Server Upgrade
Guide (820-3255) for more information.
To Install the Netra CP3260 blade server, perform the following steps:
1. If you have installed a Netra CP32x0 ARTM, go to the front of the system and
locate the card slot where you installed the ARTM at the rear of the system.
2. Remove the filler panel, if necessary.
The filler panel is secured to the card cage using two screws, one at the top of the
filler panel, the other at the bottom. Store the filler panel in a safe place; you
might need to use it again if you have to remove a card for an extended period of
time.
3. Prepare the blade server by opening the injector/ejector latches (FIGURE 2-8).
4. Carefully align the edges of the blade server with the card guides in the
appropriate slot (FIGURE 2-7).
It might be helpful to look into the enclosure to verify correct alignment of the
rails in the guides.
5. Taking care to keep the blade server aligned in the guides, slide the blade
server in until the injector/ejector latches engage the card cage.
2-20
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 2-7
Installing Blade Server Into Chassis Slot
6. Push the blade server slightly into the midplane connectors, and then close the
latches to seat the blade server in the connectors (FIGURE 2-8).
7. When the lower latch is closed, the blue Hot-Swap LED blinks while the blade
server is initializing. The blue LED turns off and the green OK LED lights when
the blade server is ready.
8. Tighten the locking screws to ensure the blade server is secured into the shelf
(FIGURE 2-8).
Chapter 2
Hardware Installation
2-21
FIGURE 2-8
2.8
Netra CP3260 Blade Server Latches and Locking Screws
Connecting External I/O Cables
External I/O cables are connected to the Netra CP3260 blade server, or to the Netra
CP32x0 ARTM when a rear transition module is used.
2-22
Netra CP3260 Blade Server User’s Guide • April 2009
Note – Shielded cables are required for serial I/O ports. Unshielded cables can be
used on Ethernet ports to satisfy EMI compliance standards. The shields for all
shielded cables must be terminated on both ends.
Information on connecting each of these cables follows:
■
For Ethernet connections, category 5e or better network cable is required. One
end of the Ethernet cable is connected to a suitable 10/100/1000BASE-T switch
and the other end to one of the Ethernet ports on the Netra CP3260 blade server.
A single 10/100BASE-T Ethernet port is also available on a Netra CP32x0 ARTM
when installed. Refer to the appropriate Netra CP32x0 ARTM User’s Guide for
more information.
Use the Ethernet device names shown in TABLE 2-1 when configuring the Ethernet
ports.
TABLE 2-1
Ethernet Device Names
Ethernet Ports
Solaris 10 OS
Device Name
Ethernet management port (on front panel)
e1000g4
■
A shielded asynchronous serial I/O cable can be attached from serial
communication devices to the RJ-45 serial port on the Netra CP3260 blade
server’s front panel.
A serial port is also provided on the Netra CP32x0 ARTM. You can use either the
serial port on the Netra CP3260 blade server or the serial port on Netra CP32x0
ARTM, but only one of the ports should be used at one time.
Once a serial cable is connected, use the tip utility on the host to establish a
full-duplex terminal connection with the Netra CP3260 blade server.
At the UNIX prompt in a command tool or shell tool, type:
# tip -9600 /dev/ttya (for serial port)
■
USB 2.0 cables can be attached from USB devices to the dual USB 2.0-compliant
ports on the Netra CP3260 blade server’s front panel.
Chapter 2
Hardware Installation
2-23
2.9
Hot-Swapping the Netra CP3260 Blade
Server
The Netra CP3260 blade server supports hot-swapping at the blade server level.
Note – Before hot-swapping the blade server, the system administrator should
gracefully shutdown the applications and operating system, and deactivate the
blade server. After the blade server is replaced, the system administrator should
activate the new blade server.
The blue Hot-Swap LED, located on the front panel of the Netra CP3260 blade server
(FIGURE 1-1), blinks when a hot-swap is initiated, and lights steadily when the blade
server is ready to be removed from the system.
Unlatching the bottom latch on the Netra CP3260 blade server initiates the hot-swap
sequence. The LED lights steadily when the blade server can be safely removed from
the system. The reverse is true when a Netra CP3260 blade server is installed into
the system. Once the Netra CP3260 blade server is installed into the system and the
bottom latch is latched, the blue Hot-Swap LED blinks until the blade server is ready
and then turns off. The green LED lights steadily when the blade server is ready.
FIGURE 2-9 shows the hot-swap latch and Hot-Swap LED.
2-24
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 2-9
Hot-Swap Latch and Hot-Swap LED
Chapter 2
Hardware Installation
2-25
2-26
Netra CP3260 Blade Server User’s Guide • April 2009
3-C HR
APTE
CHAPTER
Software Installation
This chapter contains the following sections:
3.1
■
Section 3.1, “Operating Systems and Patches” on page 3-1
■
Section 3.2, “Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and
3” on page 3-2
■
Section 3.3, “Installing as a Diskless Clients” on page 3-4
■
Section 3.4, “Firmware Updates” on page 3-8
■
Section 3.5, “Configuring for 1-GbE or 10-GbE Switches” on page 3-8
■
Section 3.6, “Downloading and Installing SunVTS Software” on page 3-9
■
Section 3.7, “Formatting the Optional Compact Flash Card” on page 3-10
Operating Systems and Patches
The Sun Netra CP3260 blade server uses the Solaris 10 8/07 OS and subsequent
compatible versions, with associated patches. Refer to the Netra CP3260 Blade Server
Product Notes (820-0455) for the Solaris OS version and the patches and patch
installation procedures. Theses notes are available online at:
http://docs.sun.com/app/docs/prod/netra.brds
The Solaris OS software and patches can be downloaded from SunSolveSM or the Sun
Download Center (SDLC) at:
http://www.sun.com/download
Note – For information on versions of the Solaris OS, including installation, see the
appropriate Solaris Documentation Collection at the Sun Documentation web site at
http://docs.sun.com/app/docs/prod/solaris.
3-1
Depending the blade server’s hardware configuration, the Sun Netra CP3260 blade
server can be used as a diskless client with the Solaris OS installed on a boot server
(see Section 3.3, “Installing as a Diskless Clients” on page 3-4. The OS can also be
downloaded to a boot device like the Netra CP32x0 SAS Storage ARTM, Dual HD, or
an optional Compact Flash.
3.2
Configuring Payload OS NIU Driver for
Multiplexing to Zones 2 and 3
The MUX configuration by system management software and payload driver
configuration must be in sync for proper functioning of both 10Gbps Ethernet
(nxge) interfaces. The payload driver relies on the driver configuration file
(nxge.conf) to initialize interfaces to either Zone 2 (backplane) or Zone 3 (ARTM).
The driver uses phy-type property to configure each interface. On the Netra
CP3260 blade server, the phy-type can have the following values:
■
phy-type=gsd (for interface to operate at 1Gbps with Zone 2 connectivity)
■
phy-type=xgsd (for interface to operate at 10Gbps with Zone 2 connectivity)
■
phy-type=xgf (for interface to operate at 10Gbps with Zone 3 connectivity)
For default installations, the nxge driver is in the /platform/sun4v/kernel/drv
directory.
The following examples show how to configure zone/link speed in the nxge driver
file for Zone 2 and Zone 3 connectivity when using the Solaris OS.
CODE EXAMPLE 3-1
Zone 2 Connectivity (Both nxge Ports)
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type
= "gsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type
= "xgsd";
Note – The phy-type=”gsd” sets the interface at 1Gbps and the phy-type=
”xgsd” sets the interface at 10Gbps.
CODE EXAMPLE 3-2
Zone 3 Connectivity (Both nxge Ports)
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type
= "xgf";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type
= "xgf";
3-2
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 3-3
Zone 2 and Zone 3 Connectivity: First NIU Port (nxge0) to Zone 2
(at 10Gbps Speed) and the Second NIU Port (nxge1) to Zone3
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type
= "xgsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type
= "xgf";
CODE EXAMPLE 3-4
Zone 2 and Zone 3 Connectivity: First NIU Port (nxge0) to Zone 3
(at 10Gbps Speed) and the Second NIU Port (nxge1) to Zone2
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type
= "xgf";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type
= "xgsd";
Chapter 3
Software Installation
3-3
3.3
Installing as a Diskless Clients
The following procedures describe how to create a boot server for diskless clients
and how to add new diskless clients to the patched boot server. For additional
instructions on installing diskless clients, refer to the appropriate Solaris
Documentation Collection at the Sun Documentation web site at:
http://docs.sun.com/app/docs/prod/solaris
You must have a superuser password on your diskless server to perform the tasks in
the following sections.
3.3.1
Creating a Boot Server for Diskless Clients
This procedure sets up a boot server by starting the operating environment services
required for diskless clients. Once you have set up the boot server, see Section 3.3.2,
“Adding a Diskless Client” on page 3-5 for instructions on adding diskless clients to
the boot server.
To create a boot server for diskless clients:
1. Verify that the IP addresses for all other network interfaces on the boot server
have corresponding host names in the hosts database.
2. Log in to the network server as superuser and change to the /usr/sadm/bin
directory.
# cd /usr/sadm/bin
3. Use the smosservice command to add boot services to the installation server.
Note – The \ (backslash) in the following code boxes is a line-continuation character
indicating that the command is continued on the next line.
# ./smosservice add -u root -p root_password -- -x mediapath=image_directory \
-x platform=sparc.sun4v.Solaris_n -x cluster=SUNWCXall -x locale=locale
Where:
3-4
■
root_password is the root password for the installation server
■
image_directory is the path to the directory where the Solaris install image is stored
■
n is the Solaris OS version you are using.
Netra CP3260 Blade Server User’s Guide • April 2009
■
locale is the locale that you want to use
■
\ (backslash) is a line-continuation character indicating that the command is
continued on the next line.
Refer to the smosservice(1M) man page for more information and options.
For example:
# ./smosservice add -u root -p root_password -- -x mediapath=/export/install \
-x platform=sparc.sun4v.Solaris_10 -x cluster=SUNWCXall -x locale=en_US
Where:
■
root_password = root_password
■
image_directory = /export/install
■
locale = en_US
■
n = 10 (for Solaris 10)
■
\ (backslash) is a line-continuation character indicating that the command is
continued on the next line.
4. Download and install additional patches.
Refer to the Netra CP3260 Blade Server Product Notes (820-0455) for the latest
information on the patches available for the Sun Netra CP3260 blade server. The
document can be downloaded from the following web site:
http://docs.sun.com/app/docs/prod/netra.brds
Follow the instructions in the Netra CP3260 Blade Server Product Notes (820-0455)
for downloading and applying patches to a diskless clients boot server.
5. After the patches are installed, follow the procedure in Section 3.3.2, “Adding a
Diskless Client” on page 3-5.
3.3.2
Adding a Diskless Client
1. Prepare a patched boot server for the diskless clients.
Follow the steps in Section 3.3.1, “Creating a Boot Server for Diskless Clients” on
page 3-4 to create a boot server for the diskless clients.
2. Log in to the patched boot server as superuser.
3. Collect the following information for the diskless client you are adding:
■
Client’s IP address
■
Client’s Ethernet address
■
Client’s host name
Chapter 3
Software Installation
3-5
4. Change directories to the /usr/sadm/bin directory.
# cd /usr/sadm/bin
5. Set up the diskless clients.
For each diskless client, type the following command as superuser:
# ./smdiskless add -- -i ip_address -e ethernet_address -n host_name \
-x os=sparc.sun4v.Solaris_n -x root=/export/root/host_name \
-x swap=/export/swap/host_name -x swapsize=swap_size -x tz=time_zone \
-x locale=locale -x ns=name_service -x nameserver=name_server
Where:
■
ip_address is the client’s IP address
■
ethernet_address is the client’s Ethernet address
■
host_name is the client’s host name
■
n is the Solaris OS version you are using
■
swap_size is the size of the swap space that you will be using. The default is 24,
however your swap space should be the same amount as your memory
■
time_zone is the client’s time zone
■
locale is the client’s locale
■
name_service is the client’s nameservice
■
name_server is the nameserver’s hostname
■
\ (backslash) is a line-continuation character indicating that the command is
continued on the next line.
Refer to the smdiskless(1M) man page for more information and options.
For example:
# ./smdiskless add -- -i 129.144.214.99 -e 8:0:20:22:b3:aa -n client_host \
-x os=sparc.sun4v.Solaris_10 -x root=/export/root/client_host \
-x swap=/export/swap/client_host -x swapsize=999 -x tz=US/Pacific \
-x locale=en_US -x ns=NIS -x nameserver=nameserver_host
Where:
3-6
■
ip_address = 129.144.214.99
■
ethernet_address = 8:0:20:22:b3:aa
■
host_name = client_host
■
n = 10 (for Solaris 10)
■
swap_size = 128
Netra CP3260 Blade Server User’s Guide • April 2009
■
time_zone = US/Pacific
■
locale = en_US
■
name_service = NIS
■
name_server = nameserver_host
■
\ (backslash) is a line continuation character indicating that the command is
continued on the next line.
You must type your superuser password again after typing this command. The
installation process should take roughly 5 minutes per client and about 15-30
minutes for the operating environment service to install; however, no progress is
displayed on screen while the process is running. Do not cancel or kill the process
until the process has successfully completed.
You should see messages similar to the following after a few minutes, confirming
that the command went through successfully the second time:
Login to client_host as user root was successful.
Download of com.sun.admin.osservermgr.cli.OsServerMgrCli from client_host was
successful.
6. Boot the diskless client.
3.3.3
Determining Local Network IP Addresses and
Host Names
Collect the following information to connect hosts to the local area network (LAN).
Ask your network administrator for help, if necessary. You can use TABLE 3-1 to
record this information. This information is not needed for a standalone installation.
TABLE 3-1
Local Network Information
Information Needed
Your Information
IP address* and host name for each Sun
Netra CP3260 client
Domain name
Type of name service and corresponding
name server names and IP addresses—for
example, DNS and NIS (or NIS+)
Subnet mask
Chapter 3
Software Installation
3-7
TABLE 3-1
Local Network Information
Information Needed
Your Information
Gateway router IP address
NFS server names and IP addresses
Web server URL
* Local IP addresses are not needed if they are assigned by a network DHCP server.
You might need the MAC (Ethernet) addresses of the local hosts to make nameserver
database entries. The MAC address can be seen in the console output while booting
to the ok prompt. It can also be derived from the host ID seen on the barcode label
(see Section 1.7.1, “Part Number, Serial Number, and Revision Number
Identification” on page 1-16).
3.4
Firmware Updates
For information on firmware updates, refer to the Netra CP3260 Blade Server Product
Notes (820-0455) which is available online at:
http://docs.sun.com/app/docs/prod/netra.brds
All Sun Netra CP3260 firmware updates can be downloaded from the Sun
Download Center (SDLC) at:
http://www.sun.com/download
3.5
Configuring for 1-GbE or 10-GbE
Switches
The Fabric interface on the Sun Netra CP3260 blade server is capable of operating at
either 1 Gbps or 10 Gbps. The Sun Netra CP3260 can be used in the Netra CT 900
server with either the Sun Netra CP3240 10-GbE switch board or the Sun Netra
CP3140 1-GbE switch board. Therefore it is important that the Solaris driver
configuration file be configured accordingly on the Netra CP3260 blade server.
3-8
Netra CP3260 Blade Server User’s Guide • April 2009
The NIU (Network Interface Unit) driver configuration file, nxge.conf, might need
to be modified to employ the proper port configurations. The nxge.conf file is
located in the /platform/sun4v/kernel/drv directory. The following examples
show the proper entries for each switch configuration. These entries are additional to
any other configuration parameters set in the nxge.conf file.
Example 1: If Sun Netra CP3240 10-GbE switches are used in Slot 7 and Slot 8, the
nxge.conf file should have the following entries:
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type = "xgsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type = "xgsd";
Example 2: If Sun Netra CP3140 1-GbE switches are used in Slot 7 and Slot 8, the
nxge.conf file should have the following entries:
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type = "gsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type = "gsd";
Example 3: If a Sun Netra CP3140 1-GbE switch is used in Slot 7 and a Sun Netra
CP3240 10-GbE switch in Slot 8, the nxge.conf file should have the following
entries:
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type = "gsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type = "xgsd";
Example 4: If a Sun Netra CP3240 10-GbE switch is used in Slot 7 and a Sun Netra
CP3140 1-GbE switch is used in Slot 8, the nxge.conf file should have the
following entries:
name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type = "xgsd";
name = "SUNW,niusl" parent = "/niu@80" unit-address = "1" phy-type = "gsd";
3.6
Downloading and Installing SunVTS
Software
Sun Validation Test Suite (SunVTS™) software is a comprehensive suite that tests
and validates the Sun Netra CP3260 blade server by verifying the configuration and
function of most hardware controllers and devices on the blade server.
Chapter 3
Software Installation
3-9
SunVTS software is used to validate a system during development, production,
inspection, troubleshooting, periodic maintenance, and system or subsystem
stressing. SunVTS software can be tailored to run on machines ranging from
desktops to servers with modifiable test instances and processor affinity features.
You can perform high-level system testing by using the appropriate version of
SunVTS software. For detailed information on SunVTS support, documentation, and
downloads, refer to the following web site:
http://www.sun.com/oem/products/vts/
You will be prompted for your Sun Online Account name and password.
For the Sun Netra CP3260 blade server, use the SunVTS6.4ps2 SPARC (or newer),
Toolkit and Documentation.
Ensure that the SunVTS software version is compatible with the Solaris OS being
used. You can find information about the SunVTS software version installed on your
system by viewing the content of the .version file:
# cat /opt/SUNWvts/bin/.version
Note – For security reasons, only a superuser is permitted to run SunVTS software.
Installation and starting instructions are included with the software when it is
downloaded.
3.7
Formatting the Optional Compact Flash
Card
The Compact Flash card is an USB removable media device on the Sun Netra CP3260
blade server. Therefore, the Solaris OS rmformat utility must be used to format the
device. The Solaris OS also provides a removable media framework for use with this
type of device.
Refer to the rmformat(1) man page for more information.
3-10
Netra CP3260 Blade Server User’s Guide • April 2009
4-C HR
APTE
CHAPTER
Firmware and Blade Server
Management
This chapter contains the following sections:
■
Section 4.1, “System Firmware” on page 4-2
■
Section 4.2, “Power-On Self-Test Diagnostics” on page 4-3
■
Section 4.3, “OpenBoot Firmware” on page 4-4
■
Section 4.4, “Error Handling Summary” on page 4-15
■
Section 4.5, “Automatic System Recovery” on page 4-16
■
Section 4.6, “Network Device Aliases” on page 4-18
■
Section 4.7, “Retrieving Device Information” on page 4-19
■
Section 4.8, “Multiplexing to Zones 2 and 3” on page 4-27
4-1
4.1
System Firmware
The Netra CP3260 blade server contains a modular firmware architecture that gives
you latitude in controlling boot initialization. You can customize the initialization,
test the firmware, and even enable the installation of a custom operating system.
This platform also employs the Intelligent Platform Management Controller
(IPMC)—described in Section 5.2.8, “Intelligent Platform Management Controller”
on page 5-13—which controls the system management, hot-swap control, and some
board hardware. The IPMC configuration is controlled by separate firmware.
The Netra CP3260 blade server boots from the 4-Mbyte system flash PROM device
that includes the power-on self-test (POST) and OpenBoot™ firmware.
A systems firmware progress sensor (SFPS) is available on the Sun Netra CP3260
blade server. The purpose of the sensor is to model the firmware running on the
payload and provide various states to the external management software (ShMM on
Netra CT 900 servers). This occurs via a standard IPMI event mechanism.
The firmware states are Progress, Hang, and Error, with various substates. The
sensor generates an IPMI event message for each state. You can verify the messages
by using clia sel command on the ShMM, through HPI event and SNMP traps for
each state of a sensor event.
For more information, see Section B.4, “Send Sensor State Command” on
page B-5.
For detailed sensor command syntax and options, refer to the Netra CT 900 Software
Developer’s Guide (819-1178). (Even if you are using a third-party chassis, the SFPS
commands and options apply, and this document is available online.)
http://docs.sun.com/app/docs/prod/n900.srvr#hic
4-2
Netra CP3260 Blade Server User’s Guide • April 2009
4.2
Power-On Self-Test Diagnostics
Power-on self-test (POST) is a firmware program that helps determine whether a
portion of the system has failed. POST verifies the core functionality of the system,
including the CPU modules, motherboard, memory, and some on-board I/O devices.
The software then generates messages that can be useful in determining the nature
of a hardware failure. POST can run even if the system is unable to boot.
If POST detects a faulty component, it is disabled automatically, preventing faulty
hardware from potentially harming any software. If the system is capable of running
without the disabled component, the system boots when POST is complete. For
example, if one of the processor cores is deemed faulty by POST, the core is disabled,
and the system boots and runs using the remaining cores.
POST diagnostic and error message reports are displayed on a console.
4.2.1
POST Test Coverage
The POST diagnostics include the following tests:
CPU Basic Test
CPU Functional Test
CPU D-Cache RAM
CPU D-Cache Tags
CPU I-Cache RAM Test
CPU I-Cache Tag RAM
CPU L2 Cache UA Array Test
CPU L2 Cache UA Array Test
CPU Interrupt Handler
CPU IMMU Memory Management Unit
CPU DMMU Memory Management Unit
FPU (Data movement and functionality)
I/O Controller register access
I/O Controller PCI configuration access
Block Memory Test
Network Interfaces (register access and basic funtionality)
Modular Arithemetic Memory parity errors Test
SPU (Security Processing Unit) Tests
Chapter 4
Firmware and Blade Server Management
4-3
4.2.2
POST Diagnostic and Error Message Format
POST diagnostic and error messages are displayed on a console. The format of the
these messages is the following:
Core-ID:Strand-ID
Core-ID:Strand-ID
Core-ID:Strand-ID
Core-ID:Strand-ID
Core-ID:Strand-ID
ERROR: TEST = test-name
H/W under test = description
Repair Instruction
MSG = error-message-body
END_ERROR
The following is an example of a POST error message
3:2>ERROR: TEST = L2-Cache Functional
3:2>H/W under test = Core l2 Cache
3:2>Repair Instructions: Replace items in order listed by ’H/W
under test’ above.
3:2>MSG = No way found to match tag address 00000000.00600000,
state 3
3:2>END_ERROR
4.3
OpenBoot Firmware
The Solaris OS installed operates at different run levels. For a full description of run
levels, refer to the Solaris system administration documentation.
Most of the time, the OS operates at run level 2 or run level 3, which are multiuser
states with access to full system and network resources. Occasionally, you might
operate the system at run level 1, which is a single-user administrative state.
However, the lowest operational state is run level 0.
When the OS is at run level 0, the ok prompt appears. This prompt indicates that the
OpenBoot firmware is in control of the system.
There are a number of scenarios under which OpenBoot firmware control can occur.
By default, before the operating system is installed, the system comes up under
OpenBoot firmware control.
4-4
■
When the auto-boot? OpenBoot configuration variable is set to false, the system
boots to the ok prompt.
■
When the operating system is halted, the system transitions to run level 0 in an
orderly way.
Netra CP3260 Blade Server User’s Guide • April 2009
4.3.1
■
When the operating system crashes, the system reverts to OpenBoot firmware
control.
■
During the boot process, when there is a serious hardware problem that prevents
the operating system from running, the system reverts to OpenBoot firmware
control.
■
When a serious hardware problem develops while the system is running, the
operating system transitions smoothly to run level 0.
■
When the OS is deliberately placed under the OpenBoot firmware control in order
to execute firmware-based commands.
Getting to the ok Prompt
There are different ways of reaching the ok prompt. The methods are not equally
desirable. See TABLE 4-1 for details.
TABLE 4-1
Ways of Accessing the ok Prompt
Access Method
What to Do
Graceful shutdown of
the Solaris OS
From a shell or command tool window, issue an appropriate
command (for example, the shutdown or init command) as
described in Solaris system administration documentation.
Manual system reset
Setting the OBP auto-boot variable to false causes the system
to stop at the ok? prompt the next time the blade server is reset.
Caution – Obtaining the ok prompt suspends all application and operating system
software. After you issue firmware commands and run firmware-based tests from
the ok prompt, the system might not be able to resume where it left off.
If possible, back up system data before starting to access the ok prompt. Also exit or
stop all applications, and warn users of the impending loss of service. For
information about the appropriate backup and shutdown procedures, see Solaris
system administration documentation.
Chapter 4
Firmware and Blade Server Management
4-5
4.3.2
Auto-Boot Options
The system firmware stores a configuration variable called auto-boot?, which
controls whether the firmware will automatically boot the operating system after
each reset. The default setting for Sun platforms is true.
Normally, if a system fails power-on diagnostics, auto-boot? is ignored and the
system does not boot unless an operator boots the system manually. An automatic
boot is generally not acceptable for booting a system in a degraded state. Therefore,
the Netra CP3260 server OpenBoot firmware provides a second setting,
auto-boot-on-error?. This setting controls whether the system will attempt a
degraded boot when a subsystem failure is detected. Both the auto-boot? and
auto-boot-on-error? switches must be set to true to enable an automatic
degraded boot. To set the switches, type:
{0} ok setenv auto-boot? true
{0} ok setenv auto-boot-on-error? true
Note – The default setting for auto-boot-on-error? is false. The system will
not attempt a degraded boot unless you change this setting to true. In addition, the
system will not attempt a degraded boot in response to any fatal nonrecoverable
error, even if degraded booting is enabled. For examples of fatal nonrecoverable
errors, see Section 4.3.4, “OpenBoot Configuration Variables” on page 4-12.
4.3.3
OpenBoot Commands
You type the OpenBoot commands at the ok prompt. The OpenBoot commands that
can provide useful diagnostic information include:
■
probe-scsi and probe-scsi-all
■
probe-ide
■
show-devs
For a complete list of OpenBoot commands and more information about the
OpenBoot firmware, refer to the OpenBoot 4.x Command Reference Manual. An online
version of the manual is included with the OpenBoot Collection AnswerBook that
ships with Solaris software.
4-6
Netra CP3260 Blade Server User’s Guide • April 2009
4.3.3.1
probe-scsi and probe-scsi-all Commands
The probe-scsi and probe-scsi-all commands diagnose problems with the
SCSI devices.
Caution – If you used the halt command or the Stop-A key sequence to reach the
ok prompt, issuing the probe-scsi or probe-scsi-all command can hang the
system.
The probe-scsi command communicates with all SCSI devices connected to
on-board SCSI controllers. The probe-scsi-all command also accesses devices
connected to any host adapters installed in PCI slots.
For any SCSI device that is connected and active, the probe-scsi and
probe-scsi-all commands display its loop ID, host adapter, logical unit number,
unique worldwide name (WWN), and a device description that includes type and
manufacturer.
The following sample output is from the probe-scsi-all command with a Netra
CP32x0 ARTM connected to the Netra CP3260 blade server.
CODE EXAMPLE 4-1
probe-scsi-all Command Output
{0} ok probe-scsi-all
This command may hang the system if a Stop-A or halt command
has been executed. Please type reset-all to reset the system
before executing this command.
Do you wish to continue? (y/n) y
/pci@0/pci@0/pci@a/pci@0/usb@4,2/storage@2
Unit 0
Disk
TUSB6250USB20 DISK DRIVE
/pci@0/pci@0/pci@8/scsi@0
MPT Version 1.05, Firmware Version 1.22.01.00
Target 2
Unit 0
Disk
SEAGATE ST914602SSUN146G0400
SASAddress 5000c5000576b0e5 PhyNum 7
Target 3
Unit 0
Disk
SEAGATE ST914602SSUN146G0400
SASAddress 5000c50005d16415 PhyNum 6
286739329 Blocks, 146 GB
286739329 Blocks, 146 GB
{0} ok
Chapter 4
Firmware and Blade Server Management
4-7
4.3.3.2
probe-ide Command
The probe-ide command communicates with all Integrated Drive Electronics (IDE)
devices connected to the IDE bus. This is the internal system bus for media devices
such as the DVD drive.
Caution – If you used the halt command or the Stop-A key sequence to reach the
ok prompt, issuing the probe-ide command can hang the system.
The following shows sample output from the probe-ide command.
CODE EXAMPLE 4-2
probe-ide Command Output
{0} ok probe-ide
Device 0 ( Primary Master )
ATA Model: FUJITSU MHV2040BH
Device 1
( Primary Slave )
ATA Model:
Device 2 ( Secondary Master )
Not Present
Device 3 ( Secondary Slave )
Not Present
4.3.3.3
show-devs Command
The show-devs command lists the hardware device paths for each device in the
firmware device tree. The following shows some sample output.
CODE EXAMPLE 4-3
show-devs Command Output
{0} ok show-devs
/pci-performance-counters@0
/ebus@c0
/pci@0
/niu@80
/cpu@3f
/cpu@3e
/cpu@3d
/cpu@3c
/cpu@3b
/cpu@3a
/cpu@39
/cpu@38
4-8
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 4-3
show-devs Command Output
/cpu@37
/cpu@36
/cpu@35
/cpu@34
/cpu@33
/cpu@32
/cpu@31
/cpu@30
/cpu@2f
/cpu@2e
/cpu@2d
/cpu@2c
/cpu@2b
/cpu@2a
/cpu@29
/cpu@28
/cpu@27
/cpu@26
/cpu@25
/cpu@24
/cpu@23
/cpu@22
/cpu@21
/cpu@20
/cpu@1f
/cpu@1e
/cpu@1d
/cpu@1c
/cpu@1b
/cpu@1a
/cpu@19
/cpu@18
/cpu@17
/cpu@16
/cpu@15
/cpu@14
/cpu@13
/cpu@12
/cpu@11
/cpu@10
/cpu@f
/cpu@e
/cpu@d
/cpu@c
/cpu@b
/cpu@a
/cpu@9
Chapter 4
Firmware and Blade Server Management
4-9
CODE EXAMPLE 4-3
show-devs Command Output
/cpu@8
/cpu@7
/cpu@6
/cpu@5
/cpu@4
/cpu@3
/cpu@2
/cpu@1
/cpu@0
/virtual-devices@100
/virtual-memory
/memory@m0,8000000
/aliases
/options
/openprom
/chosen
/packages
/ebus@c0/serial@0,ca0000
/pci@0/pci@0
/pci@0/pci@0/pci@a
/pci@0/pci@0/pci@9
/pci@0/pci@0/pci@8
/pci@0/pci@0/pci@2
/pci@0/pci@0/pci@1
/pci@0/pci@0/pci@a/pci@0
/pci@0/pci@0/pci@a/pci@0/usb@4,2
/pci@0/pci@0/pci@a/pci@0/usb@4,1
/pci@0/pci@0/pci@a/pci@0/usb@4
/pci@0/pci@0/pci@a/pci@0/usb@4,2/storage@2
/pci@0/pci@0/pci@a/pci@0/usb@4,2/storage@2/disk
/pci@0/pci@0/pci@9/network@0,1
/pci@0/pci@0/pci@9/network@0
/pci@0/pci@0/pci@8/scsi@0
/pci@0/pci@0/pci@8/scsi@0/disk
/pci@0/pci@0/pci@8/scsi@0/tape
/pci@0/pci@0/pci@2/network@0,1
/pci@0/pci@0/pci@2/network@0
/pci@0/pci@0/pci@1/network@0,1
/pci@0/pci@0/pci@1/network@0
/niu@80/network@1
/niu@80/network@0
/virtual-devices@100/rtc@5
/virtual-devices@100/console@1
/virtual-devices@100/flashprom@0
/virtual-devices@100/random-number-generator@e
/virtual-devices@100/ncp@6
/virtual-devices@100/n2cp@7
4-10
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 4-3
show-devs Command Output
/virtual-devices@100/channel-devices@200
/virtual-devices@100/channel-devices@200/virtual-channel-client@1
/virtual-devices@100/channel-devices@200/virtual-channel@0
/virtual-devices@100/channel-devices@200/virtual-channel-client@2
/virtual-devices@100/channel-devices@200/virtual-channel@3
/openprom/client-services
/packages/obp-tftp
/packages/kbd-translator
/packages/SUNW,asr
/packages/dropins
/packages/terminal-emulator
/packages/disk-label
/packages/deblocker
/packages/SUNW,builtin-drivers
4.3.3.4
Using watch-net and watch-net-all Commands to
Check the Network
The watch-net diagnostics test monitors Ethernet packets on the primary network
interface. The watch-net-all diagnostics test monitors Ethernet packets on the
primary network interface and on any additional network interfaces connected to
the system board. Good packets received by the system are indicated by a period (.).
Errors such as the framing error and the cyclic redundancy check (CRC) error are
indicated with an X and an associated error description.
●
To start the watch-net diagnostic test, type the watch-net command at the
ok prompt.
{0} ok watch-net
1000 Mbps full duplex Link up
Looking for Ethernet Packets.
‘.’ is a Good Packet. ‘X’ is a Bad Packet.
Type any key to stop.................................
●
To start the watch-net-all diagnostic test, type watch-net-all at the ok
prompt.
{0} ok watch-net-all
/pci@0/pci@0/pci@1/network@0,1
1000 Mbps full duplex Link up
Looking for Ethernet Packets.
‘.’ is a Good Packet. ‘X’ is a Bad Packet.
Type any key to stop.
.....
Chapter 4
Firmware and Blade Server Management
4-11
/pci@0/pci@0/pci@1/network@0
1000 Mbps full duplex Link up
Looking for Ethernet Packets.
’.’ is a Good Packet. ’X’ is a Bad Packet.
Type any key to stop.
.....
{0} ok
4.3.4
OpenBoot Configuration Variables
The OpenBoot configuration variables are stored in the OBP flash PROM and
determine how and when OpenBoot tests are performed. This section explains how
to access and modify OpenBoot configuration variables. For a list of important
OpenBoot configuration variables, see TABLE 4-2.
Changes to OpenBoot configuration variables take effect at the next reboot.
OpenBoot Configuration Variables
TABLE 4-2
Variable
Possible Values
Default Value
Description
ansi-terminal?
true, false
true
If true, enable ANSI terminal emulation.
auto-boot?
true, false
true
If true, boots automatically after power on
or reset.
auto-boot-on-error?
true, false
false
If true, boot automatically after system
error.
boot-command
variable-name
boot
Action following a boot command.
boot-file
variable-name
none
File from which to boot if diag-switch?
is false.
boot-device
variable-name
disk net
Device(s) from which to boot if
diag-switch? is false.
diag-switch?
true, false
false
If true:
• After a boot request, boot diag-file from
diag-device
If false:
• After a boot request, boot boot-file from
boot-device
4-12
Netra CP3260 Blade Server User’s Guide • April 2009
TABLE 4-2
OpenBoot Configuration Variables
Variable
Possible Values
Default Value
Description
error-reset-recovery
boot, none,
sync
boot
Specifies recovery action after an error reset.
Default is boot.
• none – No recovery action.
• boot – System attempts to boot.
• sync – Firmware attempts to execute a
Solaris sync callback routine.
fcode-debug?
true, false
false
If true, include name fields for plug-in
device FCodes.
input-device
keyboard,
virtual-console Power-on input device.
virtual-console
load-base
0-n
16384
Address.
local-mac-address?
true, false
true
If true, network drivers use their own
MAC address, not the server MAC address.
network-boot-arguments [protocol, ]
[key=value, ]
none
Arguments to be used by the PROM for
network booting. Defaults to an empty
string. network-boot-arguments can
be used to specify the boot protocol
(RARP/DHCP) to be used and a range of
system knowledge to be used in the process.
For further information, see the eeprom
(1M) man page or your Solaris Reference
Manual.
variable-name
none
nvramrc
Command script to execute if
use-nvramrc? is true.
oem-banner?
true, false
false
If true, use custom OEM banner.
oem-logo?
true, false
false
If true, use custom OEM logo; otherwise,
use Sun logo.
output-device
screen,
virtual-console Power-on output device.
virtual-console
screen-#columns
0-n
80
Sets number of columns on screen.
screen-#rows
0-n
34
Sets number of rows on screen.
scsi-initiator-id
0-15
7
SCSI ID of the Serial Attached SCSI
controller.
security-#badlogins
variable-name
none
Number of incorrect security password
attempts.
security-mode
none, command,
full
none
Firmware security level.
Chapter 4
Firmware and Blade Server Management
4-13
OpenBoot Configuration Variables
TABLE 4-2
Variable
Possible Values
Default Value
Description
security-password
variable-name
none
Firmware security password if
security-mode is not none (never
displayed). Do not set this directly.
ttya-mode
9600,8,n,1,-
9600,8,n,1,-
Serial management port (baud rate, bits,
parity, stop, handshake). The serial
management port only works at the default
values.
use-nvramrc?
true, false
false
If true, execute commands in NVRAMRC
during server startup.
verbosity
max, min,
none, normal
min
Controls the amount and detail of
OpenBoot output.
Default is min.
• none – Only error and fatal messages
are displayed on the system console.
• min – Notice, error, warning, and fatal
messages are displayed on the system
console.
• normal – Summary progress and
operational messages are displayed on
the system console in addition to the
messages displayed by the min setting.
• max – Detailed progress and operational
messages are displayed on the system
console.
4.3.4.1
Viewing and Setting OpenBoot Configuration Variables
●
■
Halt the server to display the ok prompt.
To display the current values of all OpenBoot configuration variables, use the
printenv command.
The following example shows a short excerpt of this command’s output.
{0} ok printenv
Variable Name
local-mac-address?
fcode-debug?
scsi-initiator-id
oem-logo?
auto-boot?
boot-command
boot-file
4-14
Value
true
false
7
false
true
boot
Netra CP3260 Blade Server User’s Guide • April 2009
Default Value
true
false
7
false
true
boot
boot-device
use-nvramrc?
error-reset-recovery
{0} ok
■
disk net
false
boot
disk net
false
boot
To set or change the value of an OpenBoot configuration variable, use the setenv
command:
{0} ok setenv error-reset-recovery none
error-reset-recovery = none
■
4.4
To set OpenBoot configuration variables that accept multiple keywords, separate
keywords with a space.
Error Handling Summary
Error handling during the power-on sequence falls into one of the following three
cases:
■
If no errors are detected by POST or OpenBoot firmware, the system attempts to
boot if auto-boot? is true.
■
If only nonfatal errors are detected by POST or OpenBoot firmware, the system
attempts to boot if auto-boot? is true and auto-boot-on-error? is true.
Nonfatal errors include the following:
■
SATA (Serial Advanced Technology Attachment) subsystem failure. In this
case, a working alternate path to the boot disk is required.
■
Ethernet interface failure.
■
Serial interface failure.
■
Memory failure. Given a failed FB-DIMM, the firmware will unconfigure the
entire logical bank associated with the failed module. Another non-failing
logical bank must be present in the system for the system to attempt a
degraded boot.
Note – If POST or OpenBoot firmware detects a nonfatal error associated with the
normal boot device, the OpenBoot firmware automatically unconfigures the failed
device and tries the next-in-line boot device, as specified by the boot-device
configuration variable.
■
If a fatal error is detected by POST or OpenBoot firmware, the system does not
boot regardless of the settings of auto-boot? or auto-boot-on-error?. Fatal
nonrecoverable errors include the following:
Chapter 4
Firmware and Blade Server Management
4-15
4.5
■
Any CPU failed
■
All logical memory banks failed
■
Flash RAM cyclical redundancy check (CRC) failure
■
Critical field-replaceable unit (FRU) PROM configuration data failure
■
Critical application-specific integrated circuit (ASIC) failure
Automatic System Recovery
Automatic system recovery (ASR) consists of self-test features and an
autoconfiguration capability to detect failed hardware components and unconfigure
them. By enabling ASR, the server is able to resume operating after certain nonfatal
hardware faults or failures have occurred.
If a component is monitored by ASR and the server is capable of operating without
it, the server automatically reboots if that component develops a fault or fails. This
capability prevents a faulty hardware component from stopping operation of the
entire system or causing the system to fail repeatedly.
If a fault is detected during the power-on sequence, the faulty component is
disabled. If the system remains capable of functioning, the boot sequence continues.
To support this degraded boot capability, the OpenBoot firmware uses the 1275
client interface (by means of the device tree) to mark a device as either failed or
disabled, creating an appropriate status property in the device tree node. The Solaris
OS does not activate a driver for any subsystem marked in this way.
As long as a failed component is electrically dormant (not causing random bus
errors or signal noise, for example), the system reboots automatically and resumes
operation while a service call is made.
Once a failed or disabled device is replaced with a new one, the OpenBoot firmware
automatically modifies the status of the device upon reboot.
Note – ASR is not enabled until you activate it (see Section 4.5.1.1, “To Enable
Automatic System Recovery” on page 4-17).
4-16
Netra CP3260 Blade Server User’s Guide • April 2009
4.5.1
Enabling and Disabling Automatic System
Recovery
The automatic system recovery (ASR) feature is not activated until you enable it.
Enabling ASR requires changing configuration variables in OpenBoot.
4.5.1.1
To Enable Automatic System Recovery
1. At the ok prompt, type:
ok setenv auto-boot true
ok setenv auto-boot-on-error? true
2. To cause the parameter changes to take effect, type:
ok reset-all
The system permanently stores the parameter changes and boots automatically
when the OpenBoot configuration variable auto-boot? is set to true (its default
value).
Note – To store parameter changes, you can also power cycle the system using the
front panel Power button.
4.5.1.2
To Disable Automatic System Recovery
1. At the ok prompt, type:
ok setenv auto-boot-on-error? false
2. To cause the parameter changes to take effect, type:
ok reset-all
The system permanently stores the parameter change.
Note – To store parameter changes, you can also power cycle the system using the
front panel Power button.
Chapter 4
Firmware and Blade Server Management
4-17
After you disable the automatic system recovery (ASR) feature, it is not activated
again until you re-enable it.
4.6
Network Device Aliases
A device alias is a shorthand representation of a device path. The Solaris OS provides
some predefined device aliases for the network devices so that you do not need to
type the full device path name. TABLE 4-3 lists the network device aliases, the default
Solaris OS device names, and associated ports for the Netra CP3260 blade server.
You can use the devalias command to display the device aliases.
TABLE 4-3
4-18
Network Device Aliases
Device Alias
Default Solaris 10 OS
Device Name
Port Description
net, net0
e1000g0
Base Interface Ethernet 0
net1
e1000g1
Base Interface Ethernet 1
net2
e1000g4
Management Ethernet (Ethernet port on front
panel)
net3
e1000g5
Management Ethernet Rear Access (Ethernet
port on ARTM)
net4
e1000g2
Rear Access (ARTM) Ethernet 0
net5
e1000g3
Rear Access (ARTM) Ethernet 1
Netra CP3260 Blade Server User’s Guide • April 2009
4.7
Retrieving Device Information
You can use the Solaris platform information and control library (PICL) framework
for obtaining the state and condition of the Netra CP3260 blade server.
The PICL framework provides information about the system configuration that it
maintains in the PICL tree. Within this PICL tree is a subtree named frutree, which
represents the hierarchy of system field-replaceable units (FRUs) with respect to a
root node in the tree called chassis. The frutree represents physical resources of the
system. The PICL tree is updated whenever a change occurs in a device’s status.
TABLE 4-4 shows the frutree entries and properties that describe the condition of the
Netra CP3260 blade server.
TABLE 4-4
PICL Frutree Entries and Description for the Netra CP3260 Board
Frutree Entry:Property
Entry Description
Example of Condition
CPU (location) :State State of the receptacle or slot
connected
CPU (fru)
:Condition
Condition of the blade server or
occupant
ok
CPU (fru) :State
State of the blade server or occupant
configured
CPU (fru) :FRUType
FRU type
bridge/fhs
The prtpicl -v command shows the condition of all devices in the PICL tree.
Sample output from the prtpicl command on the Netra CP3260 blade server is
shown in CODE EXAMPLE 4-4.
CODE EXAMPLE 4-4
prtpicl Command Output
# prtpicl
/ (picl, 6f00000001)
platform (sun4v, 6f00000005)
scsi_vhci (devctl, 6f00000022)
memory (memory, 6f000000cd)
memory-segment (memory-segment, 6f00000c10)
memory-bank (memory-bank, 6f00000c13)
memory-bank (memory-bank, 6f00000c1c)
memory-bank (memory-bank, 6f00000c25)
memory-bank (memory-bank, 6f00000c2e)
virtual-devices (virtual-devices, 6f000000df)
channel-devices (channel-devices, 6f000000f2)
virtual-channel (obp-device, 6f000000ff)
virtual-channel-client (obp-device, 6f00000109)
virtual-channel (obp-device, 6f00000113)
Chapter 4
Firmware and Blade Server Management
4-19
CODE EXAMPLE 4-4
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
4-20
prtpicl Command Output
virtual-channel-client (obp-device, 6f0000011d)
n2cp (obp-device, 6f00000127)
ncp (obp-device, 6f00000134)
random-number-generator (obp-device, 6f00000141)
flashprom (flashprom, 6f0000014d)
console (serial, 6f00000154)
rtc (obp-device, 6f00000161)
(cpu, 6f00000169)
(cpu, 6f00000177)
(cpu, 6f00000185)
(cpu, 6f00000193)
(cpu, 6f000001a1)
(cpu, 6f000001af)
(cpu, 6f000001bd)
(cpu, 6f000001cb)
(cpu, 6f000001d9)
(cpu, 6f000001e7)
(cpu, 6f000001f5)
(cpu, 6f00000203)
(cpu, 6f00000211)
(cpu, 6f0000021f)
(cpu, 6f0000022d)
(cpu, 6f0000023b)
(cpu, 6f00000249)
(cpu, 6f00000257)
(cpu, 6f00000265)
(cpu, 6f00000273)
(cpu, 6f00000281)
(cpu, 6f0000028f)
(cpu, 6f0000029d)
(cpu, 6f000002ab)
(cpu, 6f000002b9)
(cpu, 6f000002c7)
(cpu, 6f000002d5)
(cpu, 6f000002e3)
(cpu, 6f000002f1)
(cpu, 6f000002ff)
(cpu, 6f0000030d)
(cpu, 6f0000031b)
(cpu, 6f00000329)
(cpu, 6f00000337)
(cpu, 6f00000345)
(cpu, 6f00000353)
(cpu, 6f00000361)
(cpu, 6f0000036f)
(cpu, 6f0000037d)
(cpu, 6f0000038b)
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 4-4
prtpicl Command Output
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
cpu
pci
(cpu, 6f00000399)
(cpu, 6f000003a7)
(cpu, 6f000003b5)
(cpu, 6f000003c3)
(cpu, 6f000003d1)
(cpu, 6f000003df)
(cpu, 6f000003ed)
(cpu, 6f000003fb)
(cpu, 6f00000409)
(cpu, 6f00000417)
(cpu, 6f00000425)
(cpu, 6f00000433)
(cpu, 6f00000441)
(cpu, 6f0000044f)
(cpu, 6f0000045d)
(cpu, 6f0000046b)
(cpu, 6f00000479)
(cpu, 6f00000487)
(cpu, 6f00000495)
(cpu, 6f000004a3)
(cpu, 6f000004b1)
(cpu, 6f000004bf)
(cpu, 6f000004cd)
(cpu, 6f000004db)
(pciex, 6f000004e9)
pci (pciex, 6f0000050e)
pci (pciex, 6f00000533)
network (network, 6f0000055a)
network (network, 6f00000582)
pci (pciex, 6f000005aa)
network (network, 6f000005d1)
network (network, 6f000005f9)
pci (pciex, 6f00000621)
pci (pciex, 6f00000647)
network (network, 6f0000066e)
network (network, 6f00000696)
pci (pciex, 6f000006be)
pci (pci, 6f000006e4)
usb (ohci, 6f0000070e)
usb (ohci, 6f00000736)
usb (ehci, 6f0000075e)
storage (obp-device, 6f00000786)
disk (block, 6f000007b0)
ebus (ebus, 6f000007ca)
serial (serial, 6f000007db)
pci-performance-counters (obp-device, 6f000007ea)
niu (sun4v, 6f000007f5)
Chapter 4
Firmware and Blade Server Management
4-21
CODE EXAMPLE 4-4
prtpicl Command Output
network (network, 6f00000804)
network (network, 6f00000814)
os-io (console, 6f00000824)
pseudo (devctl, 6f00000843)
zconsnex (devctl, 6f0000084c)
obp (picl, 6f0000001f)
packages (packages, 6f00000034)
SUNW,builtin-drivers (SUNW,builtin-drivers, 6f0000003a)
deblocker (deblocker, 6f00000040)
disk-label (disk-label, 6f00000047)
terminal-emulator (terminal-emulator, 6f0000004d)
dropins (dropins, 6f00000054)
SUNW,asr (SUNW,asr, 6f0000005b)
kbd-translator (kbd-translator, 6f00000061)
obp-tftp (obp-tftp, 6f00000067)
ufs-file-system (ufs-file-system, 6f0000006d)
chosen (chosen, 6f00000074)
openprom (openprom, 6f00000081)
client-services (client-services, 6f0000008b)
options (options, 6f00000091)
aliases (aliases, 6f000000be)
virtual-memory (virtual-memory, 6f000000d5)
iscsi (iscsi, 6f0000082b)
physical-platform (picl, 6f00000c44)
SYS (chassis, 6f00000c47)
MB (container, 6f00000c4d)
SEEPROM (other, 6f00000c58)
SCC_NVRAM (other, 6f00000c5e)
GBE0 (container, 6f00000c64)
GBE1 (container, 6f00000c6b)
PCIE (container, 6f00000c72)
PCIE-IO (container, 6f00000c79)
CMP0 (other, 6f00000c80)
NIU0 (other, 6f00000c86)
NIU1 (other, 6f00000c8c)
PIU0 (other, 6f00000c92)
MCU0 (other, 6f00000c98)
MCU1 (other, 6f00000c9e)
MCU2 (other, 6f00000ca4)
MCU3 (other, 6f00000caa)
L2_BANK0 (other, 6f00000cb0)
L2_BANK1 (other, 6f00000cb6)
L2_BANK2 (other, 6f00000cbc)
L2_BANK3 (other, 6f00000cc2)
L2_BANK4 (other, 6f00000cc8)
L2_BANK5 (other, 6f00000cce)
L2_BANK6 (other, 6f00000cd4)
4-22
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 4-4
prtpicl Command Output
L2_BANK7 (other, 6f00000cda)
BR0 (other, 6f00000ce0)
CH0 (other, 6f00000ce6)
D0 (module, 6f00000cec)
R0 (unknown, 6f00000cf7)
R1 (unknown, 6f00000cfc)
SEEPROM (other, 6f00000d01)
PRSNT (presence-indicator, 6f00000d07)
T_AMB (temperature-sensor, 6f00000d0f)
CH1 (other, 6f00000d1f)
D0 (module, 6f00000d25)
R0 (unknown, 6f00000d30)
R1 (unknown, 6f00000d35)
SEEPROM (other, 6f00000d3a)
PRSNT (presence-indicator, 6f00000d40)
T_AMB (temperature-sensor, 6f00000d48)
BR1 (other, 6f00000d58)
CH0 (other, 6f00000d5e)
D0 (module, 6f00000d64)
R0 (unknown, 6f00000d6f)
R1 (unknown, 6f00000d74)
SEEPROM (other, 6f00000d79)
PRSNT (presence-indicator, 6f00000d7f)
T_AMB (temperature-sensor, 6f00000d87)
CH1 (other, 6f00000d97)
D0 (module, 6f00000d9d)
R0 (unknown, 6f00000da8)
R1 (unknown, 6f00000dad)
SEEPROM (other, 6f00000db2)
PRSNT (presence-indicator, 6f00000db8)
T_AMB (temperature-sensor, 6f00000dc0)
BR2 (other, 6f00000dd0)
CH0 (other, 6f00000dd6)
D0 (module, 6f00000ddc)
R0 (unknown, 6f00000de7)
R1 (unknown, 6f00000dec)
SEEPROM (other, 6f00000df1)
PRSNT (presence-indicator, 6f00000df7)
T_AMB (temperature-sensor, 6f00000dff)
CH1 (other, 6f00000e0f)
D0 (module, 6f00000e15)
R0 (unknown, 6f00000e20)
R1 (unknown, 6f00000e25)
SEEPROM (other, 6f00000e2a)
PRSNT (presence-indicator, 6f00000e30)
T_AMB (temperature-sensor, 6f00000e38)
BR3 (other, 6f00000e48)
Chapter 4
Firmware and Blade Server Management
4-23
CODE EXAMPLE 4-4
prtpicl Command Output
CH0 (other, 6f00000e4e)
D0 (module, 6f00000e54)
R0 (unknown, 6f00000e5f)
R1 (unknown, 6f00000e64)
SEEPROM (other, 6f00000e69)
PRSNT (presence-indicator, 6f00000e6f)
T_AMB (temperature-sensor, 6f00000e77)
CH1 (other, 6f00000e87)
D0 (module, 6f00000e8d)
R0 (unknown, 6f00000e98)
R1 (unknown, 6f00000e9d)
SEEPROM (other, 6f00000ea2)
PRSNT (presence-indicator, 6f00000ea8)
T_AMB (temperature-sensor, 6f00000eb0)
CORE0 (other, 6f00000ec0)
SPU (other, 6f00000ec6)
FPU (other, 6f00000ecc)
ICACHE (other, 6f00000ed2)
DCACHE (other, 6f00000ed8)
IMMU (other, 6f00000ede)
DMMU (other, 6f00000ee4)
CORE1 (other, 6f00000eea)
SPU (other, 6f00000ef0)
FPU (other, 6f00000ef6)
ICACHE (other, 6f00000efc)
DCACHE (other, 6f00000f02)
IMMU (other, 6f00000f08)
DMMU (other, 6f00000f0e)
CORE2 (other, 6f00000f14)
SPU (other, 6f00000f1a)
FPU (other, 6f00000f20)
ICACHE (other, 6f00000f26)
DCACHE (other, 6f00000f2c)
IMMU (other, 6f00000f32)
DMMU (other, 6f00000f38)
CORE3 (other, 6f00000f3e)
SPU (other, 6f00000f44)
FPU (other, 6f00000f4a)
ICACHE (other, 6f00000f50)
DCACHE (other, 6f00000f56)
IMMU (other, 6f00000f5c)
DMMU (other, 6f00000f62)
CORE4 (other, 6f00000f68)
SPU (other, 6f00000f6e)
FPU (other, 6f00000f74)
ICACHE (other, 6f00000f7a)
DCACHE (other, 6f00000f80)
4-24
Netra CP3260 Blade Server User’s Guide • April 2009
CODE EXAMPLE 4-4
prtpicl Command Output
IMMU (other, 6f00000f86)
DMMU (other, 6f00000f8c)
CORE5 (other, 6f00000f92)
SPU (other, 6f00000f98)
FPU (other, 6f00000f9e)
ICACHE (other, 6f00000fa4)
DCACHE (other, 6f00000faa)
IMMU (other, 6f00000fb0)
DMMU (other, 6f00000fb6)
CORE6 (other, 6f00000fbc)
SPU (other, 6f00000fc2)
FPU (other, 6f00000fc8)
ICACHE (other, 6f00000fce)
DCACHE (other, 6f00000fd4)
IMMU (other, 6f00000fda)
DMMU (other, 6f00000fe0)
CORE7 (other, 6f00000fe6)
SPU (other, 6f00000fec)
FPU (other, 6f00000ff2)
ICACHE (other, 6f00000ff8)
DCACHE (other, 6f00000ffe)
IMMU (other, 6f00001004)
DMMU (other, 6f0000100a)
P0 (other, 6f00001010)
P1 (other, 6f00001016)
P2 (other, 6f0000101c)
P3 (other, 6f00001022)
P4 (other, 6f00001028)
P5 (other, 6f0000102e)
P6 (other, 6f00001034)
P7 (other, 6f0000103a)
P8 (other, 6f00001040)
P9 (other, 6f00001046)
P10 (other, 6f0000104c)
P11 (other, 6f00001052)
P12 (other, 6f00001058)
P13 (other, 6f0000105e)
P14 (other, 6f00001064)
P15 (other, 6f0000106a)
P16 (other, 6f00001070)
P17 (other, 6f00001076)
P18 (other, 6f0000107c)
P19 (other, 6f00001082)
P20 (other, 6f00001088)
P21 (other, 6f0000108e)
P22 (other, 6f00001094)
P23 (other, 6f0000109a)
Chapter 4
Firmware and Blade Server Management
4-25
CODE EXAMPLE 4-4
prtpicl Command Output
P24 (other, 6f000010a0)
P25 (other, 6f000010a6)
P26 (other, 6f000010ac)
P27 (other, 6f000010b2)
P28 (other, 6f000010b8)
P29 (other, 6f000010be)
P30 (other, 6f000010c4)
P31 (other, 6f000010ca)
P32 (other, 6f000010d0)
P33 (other, 6f000010d6)
P34 (other, 6f000010dc)
P35 (other, 6f000010e2)
P36 (other, 6f000010e8)
P37 (other, 6f000010ee)
P38 (other, 6f000010f4)
P39 (other, 6f000010fa)
P40 (other, 6f00001100)
P41 (other, 6f00001106)
P42 (other, 6f0000110c)
P43 (other, 6f00001112)
P44 (other, 6f00001118)
P45 (other, 6f0000111e)
P46 (other, 6f00001124)
P47 (other, 6f0000112a)
P48 (other, 6f00001130)
P49 (other, 6f00001136)
P50 (other, 6f0000113c)
P51 (other, 6f00001142)
P52 (other, 6f00001148)
P53 (other, 6f0000114e)
P54 (other, 6f00001154)
P55 (other, 6f0000115a)
P56 (other, 6f00001160)
P57 (other, 6f00001166)
P58 (other, 6f0000116c)
P59 (other, 6f00001172)
P60 (other, 6f00001178)
P61 (other, 6f0000117e)
P62 (other, 6f00001184)
P63 (other, 6f0000118a)
V_VMEML (voltage-sensor, 6f00001190)
V_VCORE (voltage-sensor, 6f00001197)
TTYA (other, 6f0000119e)
#
For more information on the PICL framework, refer to the picld(1M) man page.
4-26
Netra CP3260 Blade Server User’s Guide • April 2009
4.8
Multiplexing to Zones 2 and 3
A multiplexer (MUX) controller and ShMM configuration is available for use on
Netra CP3260 blade servers to multiplex 10GbE network interface unit (NIU) ports
to Zone 2 (backplane) and/or to Zone 3 (ARTM).
■
MUX control configurations take effect during the blade activation following the
configuration change
■
Both NIU ports are independently configurable
■
MUX configuration is persistent across reboots, resets, and hotswaps
■
MUX configuration remains persistent for the slot, so that if a blade is replaced
with another Netra CP3260 blade server, the new blade assumes the previous
configuration
Note – The host must be configured to match the MUX configuration.
For customers using blade servers in a Netra CT 900 chassis, a complete end-to-end
solution is provided. The MUX feature is implemented through the ShMM firmware
and IPMI commands on the IPMC. These commands extend MUX configuration
access to the management software so that during blade server hot-swaps, the MUX
configuration is persistent across blade server activations and deactivations.
Customers who use Sun Netra CP3260 blade servers in a third-party chassis, which
does not have the Netra CT 900 chassis ShMM management software implemented,
can save MUX configurations in a configuration file or in a persistent storage
managed by system management software. When the system management software
detects blade server activation, it sends the command to set MUX to the
programmed state. Because management software sends the command during every
blade server activation, the configuration is persistent across blade server
deactivation and activation.
Refer to the following documentation:
■
For ShMM commands, refer to the Netra CT 900 Server Administration and Reference
Manual (819-1177).
■
For IPMI commands, refer to the Netra CT 900 Software Developer’s Guide
(819-1178).
■
For payload configuration, see Section 3.2, “Configuring Payload OS NIU Driver
for Multiplexing to Zones 2 and 3” on page 3-2.
Chapter 4
Firmware and Blade Server Management
4-27
Be aware of the following possible issues when multiplexing zones:
4-28
■
MUX and payload driver configuration could go out of sync, requiring a system
administrator to make sure they are set to the same configuration.
■
Midplane FRUID record could be corrupted when updating with the MUX
configuration.
■
Third-party shelf management software might block updates to the midplane
FRUID.
Netra CP3260 Blade Server User’s Guide • April 2009
CHAPTER
5
Hardware Functional Descriptions
This chapter contains the following sections:
5.1
■
Section 5.1, “Hardware Architecture” on page 5-1
■
Section 5.2, “Hardware Modules” on page 5-4
Hardware Architecture
The Netra CP3260 blade server is an ATCA node board based on the UltraSPARC T2
processor. It is hot-swappable to an ATCA midplane and supports dual
10/100/1000BASE-T Ethernet interfaces as Base interface, and 10-Gb XAUI or 1-Gb
SERDES Ethernet interfaces as Fabric interfaces to support a redundant Dual Star
topology.
FIGURE 5-1 is a block diagram of the Netra CP3260 blade server. The blade server
functions can be broken down into subsystems consisting of:
■
CPU and memory subsystem
■
Service processor subsystem
■
I/O subsystem
■
IPMI subsystem
■
Power subsystem
5-1
Netra CP3260 Blade Server Block Diagram
DOC
MD4832
PORT 1
X4
Console
PORT 10
X4
FRONT/REAR
SERIAL
CONSOLE
PORT 8
Midplane Zone 2
10/100/1000
MGMNT
FRONT PANEL
ETHERNET
PCI-EXP
ETHERNET
USB 2,3
1
Base
ETHERNET
0
XAUI
XAUI_RTM
SMM15
FLASH_SEL
OBP/HV
SP NVRAM
SUNFRUID
SCC_MAC
I2C
Selector
Debug
I2C
H8
Serial
Netconsole
I2C (Netconsole)
Serial
Payload
Dual
GBE
ETHERNET
(Mgmnt)
1
10/100
BASE-T
I2C Devices
XFMR
Fabric
I2C Devices
Dual
GBE
XAUI_RTM
0
TI
TUSB6250
USB-IDE
CF
Flash
BCM5397
Switch
(5 port+RGMII/GMII)
XFMR XFMR
PHY
0 Ohm
PCIe-PCI
Bridge
XFMR
PPC Ethernet
RMII
MUX
Payload
5 port
USB Hub USB 2,3
PPC
Ethernet
RMII
MUX MUX
PORT 2
PORT 9
I2C (Netconsole)
XFMR
FPGA
X8
PCI-E
SWITCH 0
X4 X4 X8
Dual
GBE
SSI
PPC Ethernet
RMII – Port 2
PPC
I2C
Serial RS232
I2C
PPC
Parallel Bus
H8
Parallel Bus
SSI
N2 JTAG
Serial RS232
PORT 0
DC Blocking
caps.
SSI
JTAG
SP SDRAM
256 MB
Payload (RS232)
XAUI
0 1
PPC
FLASH
TPM
UltraSPARC
T2
Processor
X8 PCI
EXPRESS
HOST (OBP)
FLASH
8 FBDIMM SLOTS
POWER PC
MPC885 66MHz
FBDIMM SPD[0..7]
I2C
SYS_I2C
I2C
RTM
I2C IPMC_I2C
Serial
Front board header
I2C Devices
ADM1026
IPMI FRUID
SEL EEPROM
DC-DC
Telco Daughter Card
FRONT/REAR
SERIAL CONSOLE
SERIAL
Console
FBDIMM
FPGA
FLASH
FBDIMM
BRANCH 0-3
I2C
D190 (Vcore)
Parallel Bus
I/F
FIGURE 5-1
Zone 3 RTM
IPMB
RJ45
Serial
Midplane Zone 1
NOTES:
1) Functions in orange are for Power PC
2) Functions in yellow are replacing Southbridge
5.1.1
UltraSPARC T2 Processor and Memory
The UltraSPARC T2 processor has eight SPARC physical processor cores. Each
SPARC physical processor core has full hardware support for eight strands, two
integer execution pipelines, one floating-point execution pipeline, and one memory
pipeline. The eight SPARC cores are connected through a crossbar to an on-chip
unified 4-Mbyte, 16 way associative L2 cache.
5-2
Netra CP3260 Blade Server User’s Guide • April 2009
There are four on-chip memory controllers that interface directly to FB-DIMM
memory and include eight FB-DIMM slots (one per channel, or two per memory
controller). In addition, there are an on-chip PCI-Express I/O interface and two
10-Gb Ethernet ports. The UltraSPARC T2 processor is a highly integrated processor
that implements the 64-bit SPARC V9 architecture.
5.1.2
I/O
All I/O is provided via the UltraSPARC T2 PCI-Express interface. A PLX switch
provides the PCI-E interfaces. Dual Intel NICs (or network adapters) supply the two
Ethernet interfaces to the front panel and the ARTM for management interfaces.
5.1.3
Base and Fabric Interfaces
The Base interface extends from the Intel 82571 NIC connected to the PCI-E switch
via a x4 connection. The PCI-E switch then connects to the Power PC and to the Base
interface.
The Fabric interface uses the processor’s XAUI interface and connects to the
midplane fabric interface.
5.1.4
Additional I/O
The additional I/O includes:
■
RS-232 Serial Port
■
IDE port for Compact Flash card
■
Dual USB ports
A PCI-E-to-PCI bridge provides an on-board local PCI bus to a PCI-to-USB device
that provides the USB ports. Two USB ports connect to the front panel and one USB
port connects to an USB-to-IDE bridge. The USB-to-IDE bridge supplies the IDE
interface for the Compact Flash.
5.1.5
Power PC
The RMII port on the Power PC connects to the Base interface through an Ethernet
switch.
Chapter 5
Hardware Functional Descriptions
5-3
5.1.6
ARTM Support
ARTM support is routed to the ARTM through the Zone 3 connectors. Typical ARTM
support includes:
■
One RS-232 serial console port
■
One 10/100/100BASE-T Ethernet management port
Because Netra CP32x0 Advanced RTMs (ARTM) provide additional functionality,
the following interfaces are also routed to the ARTM.
■
A x8 PCI-E interface
■
Dual 10/100/1000 SERDES Ethernet channels (from Intel NIC)
Power for the ARTMs is provided through the Zone 3 RTM power connector.
5.2
Hardware Modules
This section provides a brief description of the Netra CP3260 hardware modules.
5.2.1
UltraSPARC T2 Processor
The UltraSPARC T2 multicore processor is the basis of the Netra CP3260 blade
server. The UltraSPARC T2 processor is based on chip multithreading (CMT)
technology that is optimized for highly threaded transactional processing. The
UltraSPARC T2 processor improves throughput while using less power and
dissipating less heat than conventional processor designs. It is high performance,
highly integrated processor that implements the 64-bit SPARC V9 architecture. On
the Netra CP3260 blade server, the UltraSPARC T2 processor operates at 1.2 GHz
and contains 16 Kbytes of instruction cache per core and 8 Kbytes of data cache per
core (FIGURE 5-2).
Depending on the model purchased, the processor has six or eight UltraSPARC
cores. Each core equates to a 64-bit execution pipeline capable of running eight
threads. The result is that the 8-core processor handles up to 64 active threads
concurrently and a 6-core processor handles up to 48 active threads concurrently.
5-4
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE 5-2
UltraSPARC T2 Multicore Processor Block Diagram
Chapter 5
Hardware Functional Descriptions
5-5
5-6
Netra CP3260 Blade Server User’s Guide • April 2009
1.2
GHz
UltraSPARC T2
667
MHz
64
SPARC
core
SPARC
core
L2
Bank 1
L2
Bank 1
14
L2
Bank 0
TCU
10 Gb MAC
10 Gb MAC
14
MCU2
14
10
14
10
64
128
L2
Bank 1
CCU
10
64
L2
Bank 0
SPARC
core
14
10
64
L2
Bank 1
SPARC
core
MCU1
64
128
SPARC
core
10
10
L2
Bank 0
Cache
Crossbar
(CCX)
14
64
128
SPARC
core
MCU0
64
SPARC
core
SPARC
core
10
L2
Bank 0
128
Fully buffered
DIMMs (FBD)
4.0
GHz
MCU3
14
10
64
14
Efuse
SIU
NIU
Optional dual channel mode
PCI-E
SSI ROM I/F
5.2.1.1
Electronic Fuse
The Electronic Fuse (Efuse) block within the UltraSPARC T2 processor contains
configuration information that is electronically burned in manufacturing.
Chapter 5
Hardware Functional Descriptions
5-7
5.2.1.2
Cores
The UltraSPARC T2 processor provides 8 physical SPARC processor cores and each
physical core is capable of supporting 8 threads for a total of 64 threads.
5.2.1.3
L2 Cache
The UltraSPARC T2 provides a total of 4 Mbytes of L2 cache banked 8 ways.
5.2.1.4
Memory Controller
The UltraSPARC T2 supports 4 FB-DIMM memory controllers, each controller
capable of supporting 2 FB-DIMM memory channels. The UltraSPARC T2 connects
directly to 8 DIMMS (one DIMM per channel) providing for a total of 8 DIMM slots.
See Section 5.2.2, “Memory Subsystem” on page 5-9 for details on the Netra CP3260
memory design.
5.2.1.5
I/O Interface
The UltraSPARC T2 provides the following I/O interfaces
■
■
x8 PCI Express interface
Two 10 Gbps Ethernet (XAUI) ports
PCI Express
The UltraSPARC T2 provides a PCI Express Unit (PEU) that implements the root
complex behavior of the PCI -Express Base specification 1.0A.
The PEU supports x1, x2, x4, and x8 configuration at the data rate of 2.5Bb/s in each
direction. The PEU also supports the lane reversal feature thus easing blade server
routing restrictions.
XAUI Interface
The XAUI interface is a high speed point-to-point serial interface with four
differential pairs for transmit (TX) and four differential pairs for receive (RX),
operating at 3.125 Gbps.
5-8
Netra CP3260 Blade Server User’s Guide • April 2009
The XAUI interface is the Fabric interface to the ATCA midplane. The XAUI
interface is routed to the Zone 2 connector on the midplane (see Section A.4,
“Connectors and Pinout” on page A-3 for details on board connector pinout
assignments.
5.2.2
Memory Subsystem
The heart of the Memory subsystem is the FB-DIMM memory. Data from the
FB-DIMM memory controller is brought out on a high-speed serial bus that connects
to a single memory DIMM per channel. The FB-DIMM provides an Advanced
Memory Buffer (AMB) which converts this serial interface into the standard memory
interface used by DDR2 memory.
The UltraSPARC T2 provides 4 FB-DIMM memory controllers with each controller
supporting two memory channels.
5.2.2.1
Memory Capacity
The Netra CP3260 provides up to 8 FB-DIMM slots, two per memory controller. This
limits the maximum memory to 16 Gbytes if 2-Gbyte DIMMs are used, or 8 GBytes
if 1-Gbyte DIMMs are used.
5.2.2.2
Memory Speed
The target speed for FB-DIMMS is 667 MHz DDR2 RAMs which results in a
channeled speed of 4.0 Gbps.
5.2.3
I/O Subsystem
The Netra CP3260 I/O subsystem comprises of the following main features:
■
PCI Express switch interface
■
Dual 10/100/1000BASE-T Base interface ports to Zone 2 connector
■
Dual 1/10-Gbit XAUI ports for Fabric interface to Zone 2 connector
■
Common RTM interface via Zone 3 connector for rear access
■
Type II Compact Flash socket
■
Front panel I/O
Chapter 5
Hardware Functional Descriptions
5-9
5.2.3.1
PCI Express Switch
The Netra CP3260 uses the PLX PCI switch that encompasses a 32-lane PCI Express
switch with six configurable ports (x1, x2, x4, x8, x16).
5.2.3.2
Base Interface
The Netra CP3260 provides dual redundant 10/100/1000BASE-T Ethernet links to
connect to the midplane Base interface channels. A x4 lane port from the PCI Express
switch connects to a dual gigabit controller.
5.2.3.3
Fabric Interface
The dual XAUI ports from UltraSPARC T2 processor are multiplexed between the
midplane Zone 2 connector (that is, the fabric interface). The XAUI interface operate
at 10-Gbps or 1-Gbps.
5.2.3.4
Common ARTM
The Netra CP3260 provides a Zone 3 interface to a common ARTM which essentially
appears as an AMC card. The common ARTM supports the AMC.1 and AMC.2
compliant interfaces.
An X8 PCI Express channel is routed to the RTM slot.
The ARTM interface provides an Ethernet SERDES interface for AMC.2 compliant
functionality. These ports are sourced from the dual Gigabit Ethernet controller that
connects to the PCI Express switch using a x4 lane PCI-E channels. (See Section 5.2.4,
“Other ARTM Interfaces” on page 5-11 for other non-AMC interfaces routed to the
ARTM.)
ARTM Power and Management
The common ARTMs require two voltages:
■
3.3V STBY for management (current limited to 100 mA)
■
12V power for payload on ARTM (Maximum 25 Watts).
Power management is handled by the ATCA shelf manager’s Intelligent Platform
Management Controller (IPMC).
5-10
Netra CP3260 Blade Server User’s Guide • April 2009
5.2.4
Other ARTM Interfaces
In addition to the AMC-type interface to the ARTM, additional interfaces are routed
to the ARTM to provide for rear access to the Netra CP3260 I/O.
5.2.4.1
Serial Ports
The console port from the PPC is routed to the ARTM for rear access. When a serial
device is connected into the ARTM’s serial port, the console data is output to that
port. When the ARTM serial port is not used, console data is routed to the H8 for
netconsole or to the front panel serial port provided a serial device is connected into
the port.
5.2.4.2
Ethernet Management Port
A single 10/100/1000BASE-T Ethernet port is routed from the Intel 82671 Dual
Gigabit Ethernet controller to rear panel RJ-45 connectors. A second port goes to the
front panel RJ-45 management port.
5.2.5
Front Panel I/O
The front panel I/O consists of the following:
5.2.5.1
■
Ethernet Management port
■
Serial port
■
Dual USB ports
Ethernet Management Port
A single 10/100/1000BASE-T Ethernet port is routed from the Intel 82671 Dual
Gigabit Ethernet controller to front panel RJ-45 connector. A second management
port goes to the ARTM.
5.2.5.2
Serial Port
The serial console port from the PPC is routed to the front panel and terminated in
an RJ-45 connector.
Chapter 5
Hardware Functional Descriptions
5-11
Note – A serial port is also provided on the Netra CP32x0 ARTM. You can use
either the serial port on the front panel or the serial port on Netra CP32x0 ARTM,
but only one of the ports should be used at one time.
5.2.5.3
Dual USB Ports
Dual USB ports are 2.0 compliant and routed from the USB hub to the front panel.
5.2.6
Compact Flash Socket
The Netra CP3260 blade server provides a Type I/II compact flash socket for
removal media. The Compact Flash socket interface is derived from the USB-to-IDE
controller.
5.2.7
Service Processor MPC885
The Netra CP3260 blade server includes a MPC885 service processor subsystem used
for Logical Domains (LDoms) configuration and Hypervisor interaction, host system
reset, and boot support. The Netra CP3260 blade server uses the MPC885 to run the
vBSC firmware (on VxWorks).
The following I/O interfaces provided by MPC885 are used by the Netra CP3260
blade server:
5.2.7.1
■
100BASE-T – Connects to the Base interface switch
■
I2C – Provides interface to I2C devices required by vBSC (DIMM SPDs, TOD,
NVRAM)
■
UART1 – Provides serial interface to IPMC Payload
■
UART2 – Provides console interface
Field-Programmable Gate Array
The Field-Programmable Gate Array (FPGA) serves as a gateway between the
UltraSPARC T2 and the MPC subsystem, and provides support for the IPMC. The
FPGA provides the following functionality:
■
5-12
32-Kbyte SRAM integrated – used as Mailbox, Data Channel, and scratch pad for
POST. The SRAM is accessible both from the UltraSPARC T2 processor through
the SSI interface and from H8 through the H8 bus.
Netra CP3260 Blade Server User’s Guide • April 2009
■
H8 interface providing IPMC support.
■
MPC interface.
■
SSI Interface (UltraSPARC T2).
■
BUS Interface and Arbiter (Round Robin).
■
Bus Clock Control
■
Power sequence control of DC/DCs on board.
■
Interrupts.
■
Data Channel/Fast Mailbox Control.
The FPGA configuration is performed after an FPGA reset when the configuration is
downloaded from the PROM.
5.2.8
Intelligent Platform Management Controller
The Renesas H8S/2166 provides the IPM controller (IPMC) function on the Netra
CP3260 blade server. The IPMC provides PICMG 3.0 blade server management
functionality, and it interfaces to the host CPU through a serial interface. The IPMC
subsystem is powered from the standby power.
The IPMC is responsible for the following:
■
Dual-buffered IPMB interfaces to connect to IPMB-0
■
Serial payload interface to the host through PPC
■
IPMI_L interface to communicate with MMC on ARTMs
■
Power control of the entire Netra CP3260 blade server
■
Power and reset control of the AMC board
■
Hot-swap latch input and LED control
■
Power control
■
E-Keying control
Chapter 5
Hardware Functional Descriptions
5-13
5.2.8.1
■
Environmental monitoring
■
Access to all environmental I2C devices
■
Access to all I2C devices when MPC is reset
Intelligent Platform Management Bus
The BMR-H8S provides dual buffered Intelligent Platform Management Bus (IPMB)
interfaces to the IPMB-0 bus on the PICMG 3.0 midplane. The I2C channels on the
H8S are connected the IPMB-A and IPMB-B through the I2C buffers. The I2C buffers
allow the blade server I2C to be isolated from the midplane until the blade server is
fully seated and the I2C bus on the midplane is idle.
5.2.8.2
Interface to the PPC
The BMR-H8S provides one serial payload interface to the PPC. This interface
supports hardware flow control, RTS (Request To Send) and CTS (Clear To Send).
5.2.8.3
IPMB-L Interface
The H8S provides a local IPMI interface wired to the ARTM’s MMC (IPMB-L). The
MMC IPMI interface is isolated through FET and controlled by H8 GPIO pins.
5.2.8.4
ATCA Hot-Swap Latch
Hot-swap is supported by monitoring of the hot-swap handle switch. The handle
switch goes directly to one of the GPIO pins on the H8S (PE5).
Payload Shutdown in Response to Hot-Swap
The hardware supports both non-graceful shutdown and graceful shutdown of
payload in response to a hot-swap event. In case of non-graceful shutdown,
firmware on the IPMC disables the voltage rails before turning the blue LED on.
Optionally the IPMC can send a message across the serial payload interface to the
PPC.
5-14
Netra CP3260 Blade Server User’s Guide • April 2009
5.2.8.5
LEDs
The Netra CP3260 blade server supports three LEDs compliant with ATCA
specification:
■
Green LED – Healthy status
■
Amber LED – Fault condition
■
Blue LED – Indicates that blade server is safe for removal when on and hot
swapping in progress when blinking.
The LEDs are controlled by H8.
5.2.8.6
Power Control
The BMR-H8S is able to control (enable/disable) power rails to the payload.
5.2.8.7
System Monitor (ADM1026) and Thresholds
The Analog Devices ADM1026 is used for system monitoring functions. The
ADM1026 is interfaced with the IPMC, and the IPMC firmware is responsible for
monitoring these sensors.
Voltage Monitoring
The ADM1026 measures most of the blade server voltages. The ADM1026 has one
temperature sensor embedded in the device itself and supports two remote sensing
channels that require external diodes for temperature sensing. These two remote
sensors are used for measuring the CPU thermal diode temperature.
The ADM1026 measures the voltages listed in TABLE 5-1. The ADM1206 and H8
firmware monitors these voltages. When the voltages are within regulations, the
blade server functions with no warnings. When any voltage goes out of regulation
beyond approximately ± 7%, a Critical Warning is generated and presented to
the shelf manager.
Chapter 5
Hardware Functional Descriptions
5-15
TABLE 5-1
Voltage Sensor Thresholds
ADM1026 Generated
Critical Warning
(approx. 7%)
H8 Initiated Shutdown
(approx. 10%)
Sensor
Number
Sensor
Name
Voltage
Lower
Upper
Lower
Upper
7
12.0V
12V
11.16V
12.84V
11V
13.2V
8
5.0V
5V
4.8V
5.2V
4.78V
5.23V
9
3.3V
3.3V
3.07V
3.53V
3V
3.6V
10
3.3V STBY
3.3V
3.07V
3.53V
2.97V
3.63V
11
3V VBAT/STBY
3V
2.79V
No UC
no
shutdown
no
shutdown
12
1.0V
1V
0.93V
1.07V
0.9V
1.1V
13
1.1V CPU
1.1V
1.02V
1.18V
0.99V
1.21V
14
VDD 1.1V
1.1V
1.02V
1.18V
0.99V
1.21V
15
1.5V
1.5V
1.4V
1.61V
1.35V
1.65V
16
VDD 1.8V
1.8V
1.67V
1.93V
1.62V
1.98V
17
VDD 2.5V
2.5V
2.37V
2.68V
2.36V
2.75V
18
VDD_IO 1.2V
1.5V
1.4V
1.61V
1.35V
1.65V
Temperature Monitoring
The ADM1026 also monitors the CPU diode temperature on the Netra CP3260 blade
server. The ADM1206 and H8 firmware reports a minor, major, or critical alarm
when the temperature of the CPU goes beyond the thresholds listed in TABLE 5-2.
Also, if the temperature of the CPU goes beyond the Emergency H8 Shutdown
value listed in TABLE 5-2, the H8 initiates a event/warning blade server to the shelf
manager and shuts down.
TABLE 5-2
5-16
CPU Temperature Alarms
Sensor
Number
Sensor
Name
Minor Alarm
(UNC)
Major Alarm
(UC)
Critical Alarm
(UNR)
Emergency
H8 Shutdown
4
CPU Temp 1
>80° C
>90° C
> 102° C
> 102° C
5
CPU Temp 2
>80° C
>90° C
> 102° C
> 102° C
6
Board Temp
>60° C
>70° C
> 88° C
> 88° C
Netra CP3260 Blade Server User’s Guide • April 2009
The H8 temperature alarm equivalents are:
Minor Alarm = Upper Non Critical (UNC)
Major Alarm = Upper Critical (UC)
Critical Alarm = Upper Non Recoverable (UNR)
Emergency H8 Shutdown (EMR) = UNR
Caution – These voltage and temperature thresholds should not be changed under
normal operating conditions.
5.2.8.8
FRUID PROMs
There are two FRUID PROMs on the Sun Netra CP3260 and both are 64-Kbyte
SEEPROMs. One contains Sun FRU information and the other contains IPMI FRU
information. Each device stores static information, such as the part number and
manufacturing date, and dynamic data that can be updated by the host system, such
as operational statistics and failure information.
5.2.9
I/O Subsystem Resets
The UltraSPARC T2 provides a PCI Express reset out on the PEX_RESET_L pin. This
pin connects to the FPGA and in turn, this is used by the FPGA to reset the I/O
subsystem which includes the PLC PCI Express switch.
The PCIe switch in turn provides an inband reset (output pin) which, via the FPGA,
resets the:
■
Dual GbE for the Base
■
Dual GbE for RTM
■
GbE for management ports
■
RTM PCIe reset
■
Reset to PCI-E-to-PCI bridge
In addition to the processor resetting the I/O subsystem, each I/O component has
reset software write-able by the PPC to reset individual components.
The Ethernet switch is reset-able via software by the PPC. This allows the Ethernet
path between the PPC and the Base to remain active in the event the processor is
being reset.
Chapter 5
Hardware Functional Descriptions
5-17
A reset mask register is provided in the FPGA to allow the masking of resets to
individual I/O components. For example, a PEX_RESET_L reset from the
UltraSPARC T2 processor resets only those I/O subcomponents not masked by the
reset mask register.
5.2.10
ATCA Power Module (−48V to 12V)
The Netra CP3260 blade server uses the Artesyn ATCA power module solution. The
Artesyn power module provides an integrated ATCA power solution that meets
PICMG 3.0 requirements, including dual bus input, DC isolation, hold up, hot-plug,
and management power (3.3V standby). It provides a 12V intermediate bus as
backend power. Some of the salient features of the module are:
5.2.11
■
210 watts output power.
■
Input-to-output isolation.
■
12V with current rating of 17.5A.
■
3.3V standby at 1.82A. (The IPMC draws power from IPMI Power (3.3V_STBY) so
it can remain functional even if back-end logic is powered down.)
■
Isolated remote ON/OFF.
■
Isolated “A” and “B” bus detect signals.
■
High efficiency, typically 88 percent.
■
Operating input voltage: −39V to −72V.
■
The I2C interface monitors the status of fuse, input voltages, output voltages, and
temperature and sends an alarm if any of the parameters are outside the
programmable threshold.
■
FRU information.
TOD Clock Battery
The TOD clock battery must be type CR1632, with a minimum of 4ma abnormal
charging current rating (for example; a Renata CR1632).
Caution – Risk of explosion if battery is replaced by an incorrect type.
Dispose of batteries properly in accordance with manufacturer’s instructions and
local regulations.
See Section 2.5.3, “Adding or Replacing TOD Clock Battery” on page 2-14 for battery
location and installation instructions.
5-18
Netra CP3260 Blade Server User’s Guide • April 2009
APPENDIX
A
Physical Characteristics
Specifications for the Netra CP3260 blade server are provided in the following
sections:
A.1
■
Section A.1, “Form Factor” on page A-1
■
Section A.2, “Layout” on page A-1
■
Section A.3, “Front Panel” on page A-3
■
Section A.4, “Connectors and Pinout” on page A-3
Form Factor
The Netra CP3260 blade server is a standard 8U form factor, a single-slot-wide. It
complies with the blade server mechanical dimensions required by the PICMG 3.0
R1.0 Specification:
A.2
■
322.25 mm x 280 mm (length x width)
■
1.2-inch-wide front panel
Layout
The Netra CP3260 blade server layout is shown in FIGURE A-1.
A-1
FIGURE A-1
Netra CP3260 Blade Server Layout
④
⑧
①
⑦
②
③
⑥
⑤
Figure Legend
A-2
1
Ethernet port connector (RJ-45)
5
Zone 1 power connector
2
Dual USB port connector (USB 2.0)
6
Zone 2 I/O connectors
3
Serial port connector
7
Zone 3 RTM I/O connectors
4
Compact flash connector
8
Zone 3 RTM power connector
Netra CP3260 Blade Server User’s Guide • April 2009
A.3
Front Panel
The single-slot-wide, 8U front panel was designed to meet PICMG 3.0 R 1.0 and
other specifications.
A.3.1
Visual Indicators
The Netra CP3260 blade server has the following indicators on the front panel:
■
Green LED – Board’s healthy status or user programmable (ACTIVE).
■
Amber LED – Board’s fault condition (FAULT).
■
Blue LED – Indicates safe removal (hot-swap activity).
The front panel’s Ethernet ports do not have LED indicators.
A.3.2
Ports
The Netra CP3260 blade server has the following ports on the front panel:
A.4
■
One 10/100/1000BASE-T Ethernet port (RJ-45)
■
Dual USB ports (USB 2.0)
■
One serial port (RJ-45)
Connectors and Pinout
FIGURE A-1 shows all the basic I/O connectors to the front and the rear of the Netra
CP3260 blade server.
A.4.1
Front Panel Connectors
The front panel has the following connectors:
■
One 10/100/1000BASE-T Ethernet port (RJ-45)
■
One serial port (RJ-45)
■
Dual USB ports (USB 2.0)
Appendix A
Physical Characteristics
A-3
A.4.1.1
Ethernet Port
The Ethernet connector is a RJ-45 connector. The controller autonegotiates to either
10BASE-T, 100BASE-T, or 1000BASE-T. The Ethernet connector pin numbering is
shown in FIGURE A-2.
FIGURE A-2
Ethernet RJ-45 Connector
...
<
>
12345678
Link LED (green)
Activity LED (yellow)
TABLE A-1 shows the Ethernet connector pin assignments.
TABLE A-1
A.4.1.2
Ethernet Port Connector Pin Assignments
Pin
Signal
Name
Function
Pin
Signal
Name
Function
1
BI_DA+
Bi-directional pair +A
5
BI_DC−
Bi-directional pair +C
2
BI_DA−
Bi-directional pair −A
6
BI_DB−
Bi-directional pair −C
3
BI_DB+
Bi-directional pair +B
7
BI_DD+
Bi-directional pair +D
4
BI_DC+
Bi-directional pair −B
8
BI_DD−
Bi-directional pair −D
Dual USB Port
The dual-port Universal Serial Bus (USB) connector pin numbering is shown in
FIGURE A-3.
A-4
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE A-3
Dual USB Connector
1
2
3
4
3
4
B
1
2
A
For USB connector signals, see TABLE A-2.
TABLE A-2
A.4.1.3
USB Connector Pin Assignments
Pin
Signal Description
Pin
Signal Description
A1
+5 V (fused)
B1
+5 V (fused)
A2
USB2−
B2
USB3−
A3
USB2+
B3
USB3+
A4
Ground
B4
Ground
Serial Port
FIGURE A-4 contains the connector pin assignments for the front panel serial port.
Appendix A
Physical Characteristics
A-5
FIGURE A-4
Front Panel Serial Port Diagram
I0I0I
12345678
TABLE A-3 shows the serial port connector pin assignments.
TABLE A-3
A.4.2
Serial Port RJ-45 Connector Pinouts
Pin
Signal
Name
Function
Pin
Signal
Name
Function
1
RTS
Request To Send
5
GND
Ground
2
DTR
Data Terminal Ready 6
RXD
Receive Data
3
TXD
Transmit Data
7
DSR
Data Set Ready
4
GND
Ground
8
CTS
Clear To Send
Compact Flash Connector
The Compact Flash connector is a type I/II connector.
A.4.3
Midplane Power Connector (Zone 1)
The Netra CP3260 blade server uses a Zone 1 power connector. It provides support
for the following signals:
■
Two −48 volt DC power feeds (four signals each; eight signals total)
■
Two IPMB ports (two signals each, four signals total)
■
Geographic address (eight signals)
The analog test and ring voltage pins are left unconnected.
A-6
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE A-5 shows the pin assignments.
FIGURE A-5
Power Distribution Connector (Zone 1) P10
TABLE A-4 lists the power connector pin assignments.
TABLE A-4
Power Distribution Connector Pin Assignments
Pin Number Name
Description
1
Reserved
Reserved
2
Reserved
Reserved
3
Reserved
Reserved
4
Reserved
Reserved
5
HA0
HA0 Hardware Address Bit 0
6
HA1
HA1 Hardware Address Bit 1
7
HA2
HA2 Hardware Address Bit 2
8
HA3
HA3 Hardware Address Bit 3
9
HA4
HA4 Hardware Address Bit 4
10
HA5
HA5 Hardware Address Bit 5
11
HA6
HA6 Hardware Address Bit 6
12
HA7/P
HA7/P Hardware Address Bit 7(Odd Parity Bit)
13
SCL_A
IPMB Clock, Port A
14
SDA_A
IPMB Data, Port A
15
SCL_B
IPMB Clock, Port B
16
SDA_B
IPMB Data, Port B
Appendix A
Physical Characteristics
A-7
TABLE A-4
Power Distribution Connector Pin Assignments (Continued)
Pin Number Name
A.4.4
Description
17
Unused
18
Unused
19
Unused
20
Unused
21
Unused
22
Unused
23
Unused
24
Unused
25
SHELF_GND
Shelf Ground
26
LOGIC_GND
Logic Ground
27
ENABLE_B
Enable B
28
VRTN_A
Voltage Return A
29
VRTN_B
Voltage Return B
30
EARLY_A
−48V Early A
31
EARLY_B
–48V Early B
32
ENABLE_A
Enable A
33
−48V_A
−48V A
34
−48V_B
−48V B
Data Transport Connector (Zone 2)
The data transport connector consists of one 120-pin HM-Zd connector, labeled P23,
with 40 differential pairs. This is called the Zone 2 connector.
The Zone 2 connector provides the following signals:
■
Two 1000BASE-T Ethernet Base interface channels
■
Two 1-Gb SERDES on the Fabric (extended) interface
■
10-Gb XAUI on the Fabric (extended) interface
■
Synchronization Clock interface supporting CLK1A/B and CLK2A/2B
FIGURE A-6 shows the Zone 2 connectors.
A-8
Netra CP3260 Blade Server User’s Guide • April 2009
FIGURE A-6
Zone 2 Connectors
J20
J23
TABLE A-5 gives the Zone 2 J23 connector pin assignments.
TABLE A-5
Row
Zone 2 J23 Connector Pin Assignments
A
B
C
D
E
F
G
H
1
P23_F2_TX2+
P23_F2_TX2−
P23_F2_RX2+ P23_F2_RX2− P23_F2_TX3+
P23_F2_TX3−
P23_F2_RX3+
P23_F2_RX3−
2
P23_F2_TX0+
P23_F2_TX0−
P23_F2_RX0+ P23_F2_RX0− P23_F2_TX1+
P23_F2_TX1−
P23_F2_RX1+
P23_F2_RX1−
3
P23_F1_TX2+
P23_F1_TX2−
P23_F1_RX2+ P23_F1_RX2− P23_F1_TX3+
P23_F1_TX3−
P23_F1_RX3+
P23_F1_RX3−
4
P23_F1_TX0+
P23_F1_TX0−
P23_F1_RX0+ P23_F1_RX0− P23_F1_TX1+
P23_F1_TX1−
P23_F1_RX1+
P23_F1_RX1−
5
P23_BI_DA1+ P23_BI_DA1− P23_BI_DB1+ P23_BI_DB1− P23_BI_DC1+ P23_BI_DC1− P23_BI_DD1+ P23_BI_DD1−
6
P23_BI_DA2+ P23_BI_DA2− P23_BI_DB2+ P23_BI_DB2− P23_BI_DC2+ P23_BI_DC2− P23_BI_DD2+ P23_BI_DD2−
7
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
8
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
9
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
10
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
F
G
H
TABLE A-6 gives the Zone 2 J20 connector pin assignments.
TABLE A-6
Row
Zone 2 J20 Connector Pin Assignments
A
B
C
D
E
1
CLK1A_P
CLK1A_N
CLK1B_P
CLK1B_N
CLK2A_P
CLK2A_N
CLK2B_P
CLK2B_N
2
N/C
N/C
N/C
N/C
CLK3A_P
CLK3A_N
CLK3B_P
CLK3B_N
3
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
Appendix A
Physical Characteristics
A-9
TABLE A-6
Row
A
Zone 2 J20 Connector Pin Assignments (Continued)
B
C
D
E
F
G
H
4
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
5
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
6
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
7
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
8
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
9
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
10
N/C
N/C
N/C
N/C
N/C
N/C
N/C
N/C
For information about multiplexing to Zone 2 or Zone 3, see Section 4.8,
“Multiplexing to Zones 2 and 3” on page 4-27.
A.4.5
Sun Netra ARTM Connectors (Zone 3)
The Netra CP3260 blade server provides all the I/O and power connections for rear
access through the Zone 3 RTM connectors. The Zone 3 I/O connectors are shown in
FIGURE A-7 and the Zone 3 power connector is shown in FIGURE A-8. The pin numbers
and signal assignments are listed in the associated tables.
FIGURE A-7
Zone 3 Signal Connectors
J31
A-10
J32
Netra CP3260 Blade Server User’s Guide • April 2009
J33
TABLE A-7 gives the Zone 3 J31 connector signals and pin assignments.
Zone 3 J31 Connector Pin Assignments
TABLE A-7
Row
A
B
C
D
E
F
1
Z3_F1_TX1+
Z3_F1_TX1−
Z3_F1_RX0+
Z3_F1_RX0−
Z3_F1_TX0+
Z3_F1_TX0−
2
Z3_F1_RX2+
Z3_F1_RX2−
Z3_F1_TX2+
Z3_F1_TX2−
Z3_F1_RX1+
Z3_F1_RX1−
3
Z3_F2_TX0+
Z3_F2_TX0−
Z3_F1_RX3+
Z3_F1_RX3−
Z3_F1_TX3+
Z3_F1_TX3−
4
Z3_F2_RX1+
Z3_F2_RX1−
Z3_F2_TX1+
Z3_F2_TX1−
Z3_F2_RX0+
Z3_F2_RX0−
5
Z3_F2_TX3+
Z3_F2_TX3−
Z3_F2_RX2+
Z3_F2_RX2−
Z3_F2_TX2+
Z3_F2_X2−
6
N/U
N/U
N/U
N/U
Z3_F2_RX3+
Z3_F2_RX3−
7
N/U
N/U
N/U
N/U
N/U
N/U
8
N/U
N/U
N/U
N/U
N/U
N/U
9
N/U
N/U
N/U
N/U
N/U
N/U
10
N/U
N/U
N/U
N/U
N/U
N/U
N/U = not used
TABLE A-8 gives the Zone 3 J32 connector signals and pin assignments
TABLE A-8
Row
Zone 3 J32 Connector Pin Assignments
A
B
C
D
E
F
1
XAUI_MDIO
XAUI_MDC
N/U
N/U
N/U
N/U
2
N/U
N/U
N/U
N/U
SER_RJ45_RTS
SER_RJ45_DTR
3
N/U
N/U
N/U
N/U
SER_RJ45_TXD
SER_RJ45_RXD
4
N/U
N/U
N/U
N/U
SER_RJ45_DSR
SER_RJ45_CTS
5
N/U
N/U
N/U
N/U
N/C
N/C
6
RTM_LAN0_A+ RTM_LAN0_A− RTM_TXFRMR_ RTM_TXFRMR_ RTM_LAN0_B+ RTM_LAN0_B−
VOLTAGE
VOLTAGE
7
RTM_LAN0_C+ RTM_LAN0_C− RTM_ACT_
LED_N
RTM_LINK_
LED_N
RTM_LAN0_D+ RTM_LAN0_D−
8
RTM_SLAN_TX RTM_SLAN_
1+
TX1−
RTM_SLAN_
RX1+
RTM_SLAN_
RX1−
N/C
N/C
9
RTM_SLAN_TX RTM_SLAN_
2+
TX2−
RTM_SLAN_
RX2+
RTM_SLAN_
RX2−
N/U
N/U
10
N/U
N/U
N/U
N/U
N/U
N/C = not connected
N/U
N/U = not used
Appendix A
Physical Characteristics
A-11
TABLE A-9 gives the Zone 3 J33 connector signals and pin assignments.
TABLE A-9
Row
Zone 3 J33 Connector Pin Assignments
A
B
C
D
E
F
1
RTM_TX0+
RTM_TX0−
RTM_RX0+
RTM_RX0−
RTM_CON_
REF CLK+
RTM CON_
REF CLK−
2
RTM_TX1+
RTM_TX1−
RTM_RX1+
RTM_RX1−
TCLKA+
TCLKA−
3
RTM_TX2+
RTM_TX2−
RTM_RX2+
RTM_RX2−
TCLKB+
TCLKB−
4
RTM_TX3+
RTM_TX3−
RTM_RX3+
RTM_RX3−
TCLKC+
TCLKC−
5
RTM_TX4+
RTM_TX4−
RTM_RX4+
RTM_RX4−
TCLKD+
TCLKD−
6
RTM_TX5+
RTM_TX5−
RTM_RX5+
RTM_RX5−
N/U
N/U
7
RTM_TX6+
RTM_TX6−
RTM_RX6+
RTM_RX6−
N/U
N/U
8
RTM_TX7+
RTM_TX7−
RTM_RX7+
RTM_RX7−
N/U
FPGA_PRESET_
RTM−
9
N/C
N/C
N/C
N/C
RTM_MMC_L
PCI_CFG
10
N/U
N/U
N/U
N/U
GND
RTM_CON_EN_L
N/C = not connected
N/U = not used
FIGURE A-8
Zone 3 Power Connector
Zon
Pow
Con
TABLE A-10 gives the Zone 3 power connector signals and pin assignments.
A-12
Netra CP3260 Blade Server User’s Guide • April 2009
TABLE A-10
Zone 3 Power Connector Pin Assignments
R
o
w Pin 1
Pin 2
Pin 3
Pin 4
Pin 5
Pin 6
A Logic_GND
Logic_GND
Logic_GND
Shelf_GND
Shelf_GND
Shelf_GND
B Logic_GND
Logic_GND
Logic_GND
+3.3V_STBY_RTM
+3.3V_STBY_RTM
+3.3V_STBY_RTM
C I2C_RTMCONN_SCL I2C_RTMCONN_SCL I2C_RTMCONN_SCL I2C_RTMCONN_SDA I2C_RTMCONN_SDA I2C_RTMCONN_SDA
D +12V_RTM
E
RTM_PS1_L
+12V_RTM
+12V_RTM
+12V_RTM
+12V_RTM
+12V_RTM
RTM_PS1_L
RTM_PS1_L
N/C
N/C
N/C
For information about multiplexing to Zone 2 or 3, see Section 4.8, “Multiplexing to
Zones 2 and 3” on page 4-27.
A.4.6
TOD Clock Battery Holder
FIGURE A-9 shows the location of the TOD clock battery and holder.
The TOD battery must be type CR1632, with a minimum of 4ma abnormal charging
current rating (for example; a Renata CR1632).
Caution – Risk of explosion if battery is replaced by an incorrect type.
Dispose of batteries properly in accordance with manufacturer’s instructions and
local regulations.
To install the battery, slide the battery into the holder with the side labeled “+ “
facing up.
Appendix A
Physical Characteristics
A-13
FIGURE A-9
TOD Battery Location
①
②
Figure Legend
A-14
1
TOD battery location
2
TOD clock battery location with FB-DIMM removed
Netra CP3260 Blade Server User’s Guide • April 2009
APPENDIX
B
Sun OEM IPMI Commands
This appendix contains the following sections:
■
Section B.1, “Get Version Command” on page B-2
■
Section B.2, “Get RTM Status Command” on page B-3
■
Section B.3, “Solaris OS Graceful Shutdown Commands” on page B-4
■
Section B.4, “Send Sensor State Command” on page B-5
The commands described in this appendix are specific to Netra CP3260 blade server
designed by Sun Microsystems. The Internet Assigned Numbers Authority (IANA)
number assigned to Sun Microsystems is 42.
Refer to http://www.iana.org/assignments/enterprise-numbers for more
information about IANA number assignments.
The netfunction (NetFn) used for these commands is 0x2E, which is the OEM
netfunction as defined in the IPMI specification. For this netfunction, the first three
data bytes in the request packet must be this IANA number and the first three bytes
in the response packet following the completion code are the IANA number. For Sun
ATCA node boards, these three bytes are 00 00 2A.
B-1
B.1
Get Version Command
Get Version returns the IPM controller (IPMC) firmware version and Standby
CPLD version. Bytes 8, 9, and A are reserved for future use.
Command
NetFn
Opcode
Reference
Get_Version
0x2E (OEM)
0x80
--
TABLE B-1
Get Version Command Data Bytes
Type
Byte
Data Field
Request data
Byte1
00
Byte2
00
Byte3
2A
Byte1
Completion code:
00 = OK
C1 = Command not supported
CC = Invalid data in request
(Refer to IPMI specification for more completion
codes)
Byte2
00
Byte3
00
Byte4
2A
Byte5
CPLD version
Byte6
REV1 byte of IPMC firmware
Byte7
REV2 byte of IPMC firmware
Byte8
Reserved for future use (ignore)
Byte9
Reserved for future use (ignore)
ByteA
Reserved for future use (ignore)
Response data
B-2
Netra CP3260 Blade Server User’s Guide • April 2009
Example (Terminal Mode):
[B8 00 80 00 00 2A] <-------Request
[BC 00 80 00 00 00 2A 02 02 00 00 00 00] <----Response
■
The IPMC version is read as:
lower nibble of REV1 . high nibble of REV2 . low nibble of REV2
In the preceding example, the IPMC version is 2.0.0.
■
The CPLD version is read as:
-> lower nibble of CPLD version byte
In the example, the CPLD version is 2.
B.2
Get RTM Status Command
You can use the Get RTM Status command to detect the presence of a rear
transition module (RTM) in the system.
Command
NetFn
Opcode
Reference
Get_RTM_Status
0x2E (OEM)
0x88
CPLD Specification
TABLE B-2
Get RTM Status Command Data Bytes
Type
Byte
Data Field
Request data
Byte1
00
Byte2
00
Byte3
2A
Byte1
Completion code:
00 = OK
C1 = Command not supported
CC = Invalid data in request
Byte2
00
Byte3
00
Byte4
2A
Byte5
RTM presence
Bits 7 to 1 = 0
Bits 0
= RTM presence (0 = RTM not detected,
1 = RTM detected)
Response data
Appendix B
Sun OEM IPMI Commands
B-3
Example (Terminal Mode):
[B8 00 88 00 00 2A] <------Request
[BC 00 88 00 00 00 2A 01] <------Response
B.3
Solaris OS Graceful Shutdown
Commands
Users can change the graceful shutdown time out of Solaris OS with an IPMI
command executed from ShMM or a user application. Note that this setting is not
persistent; the Solaris FSM will overwrite it after every Solaris OS boot.
SUNWctfsm is the Solaris FSM for ATCA graceful shutdown and reboot. For
installation and requirements, refer the README file included in the release
package and the Sun Netra CP3260 Blade Server Product Notes (820-0455).
The commands and parameters are described in the following table.
TABLE 1
Solaris OS Graceful Shutdown Parameters in /etc/fsmd.conf
Parameter
Description and Parameters
GS_SHUTDOWN_TIMEOUT
Sets the payload graceful shutdown timeout in increments
of 100 milliseconds. The legal value range is from 0 to
0xffff (0 second to 6553 seconds).
The default value is 1200, which is 120 seconds. This 1200
will overwrite the default value 300 in IPMC.
Solaris might panic during graceful shutdown if this
parameter value is too short. Conversely, if this value is
too long, the shutdown could take too long. See Note that
follows this table.
GR_COMPLETION_NOTICE
Determines if Solaris FSM will send a reboot succeed
notice to IPMC. The legal values are yes and no.
The default value is no. IPMC might or might not
support/use it.
GR_TIMER_DISARM
Determines if Solaris FSM will disarm graceful reboot
timer. The legal values are yes and no.
The default value is no. Set to yes only if the blade server’s
firmware has no graceful reboot support.
Any parameter changes you make will become effective only after you execute the
Solaris command svcadm restart fsm.
B-4
Netra CP3260 Blade Server User’s Guide • April 2009
Note – As of the R3U2 release, Netra CP3260 sysfw 7.2.1 has graceful shutdown
support.
Note – To use the graceful shutdown effectively, users must determine a
component’s shutdown time “cost,” which depends on hardware, software, and
configuration. For example, the default 120 seconds could be too short for a Sun
Netra CP3260 blade server with a heavy I/O configuration. Conversely, it could be
too long for a Sun Netra CP3220 blade server. Ignorance of the shutdown time could
cause a panic during graceful shutdown.
For more information, refer to the following documentation:
■
Netra CT 900 Server Administration and Reference Manual (819-1177) for ShMM
commands.
■
Netra CT 900 Software Developer’s Guide (819-1178) for IPMI commands.
Even if you are using a third-party chassis, the commands and options apply, and
these documents are available online:
http://docs.sun.com/app/docs/prod/n900.srvr#hic
B.4
Send Sensor State Command
Send Sensor State is used by the firmware progress sensor or any other sensor
that intends to send sensor states to the IPM controller (IPMC). The IPMC uses this
information to put a sensor in the correct state and send the event data to the system
event log.
Command
NetFn
Opcode
Reference
Get_Version
0x2E (OEM)
0x80
--
TABLE B-3
Send Sensor State Command Data Bytes
Type
Byte
Data Field
Request data
Byte1
00
Byte2
00
Byte3
2A
Appendix B
Sun OEM IPMI Commands
B-5
TABLE B-3
Type
B-6
Send Sensor State Command Data Bytes (Continued)
Byte
Data Field
Byte4
Sensor type byte
Indicates event class or type of sensor that generates
the event message. (Refer to IPMI specification for
sensor type codes.)
Byte5
Sensor # (optional); Write to 0
Unique number representing the sensor within the
management controller that generates the event
message. The BIOS, OBP, ILOM, and OS does not
have to know and send the sensor number for the
sensors that are maintained and reported by the
IPMC.
Reserved for future use.
Byte6
EventDir | EventType
• 1 bit for EventDir, which indicates the event
transition direction. A value of 0 means assertion
event; a value of 1 means deassertion event.
• 7 bits for EventType, which indicates the type of
threshold crossing or state transition (trigger) that
produces the event. Encode using the
Event/Reading Type code. (Refer to IPMI
specification for type codes.)
Byte7
Sensor offset
Tells the sensor generic or sensor specific offset as
defined in the IPMI specification. IPMC uses this data
to define the state of the sensors. (Refer to the IPMI
specification.)
Byte8
Event Data 1
Represents the remainder of the event message data
according to the class of the event type for the sensor
(threshold, discrete, or OEM). (Refer to the IPMI
specification.)
Byte9
Event Data 2
Represents the remainder of the event message data
according to the class of the event type for the sensor
(threshold, discrete, or OEM). (Refer to the IPMI
specification.)
Byte10
Event Data 3
Represents the remainder of the event message data
according to the class of the event type for the sensor
(threshold, discrete, or OEM). (Refer to the IPMI
specification.)
Byte11
Reserved for future use. (Write zero)
Netra CP3260 Blade Server User’s Guide • April 2009
TABLE B-3
Send Sensor State Command Data Bytes (Continued)
Type
Response data
Byte
Data Field
Byte12
Reserved for future use. (Write zero)
Byte1
Completion code:
00 = OK
C1 = Command not supported
CC = Invalid data in request
(Refer to IPMI specification for more completion
codes)
Byte2
00
Byte3
00
Byte4
2A
Appendix B
Sun OEM IPMI Commands
B-7
B-8
Netra CP3260 Blade Server User’s Guide • April 2009
Index
Advanced Rear Transition Module (ARTM), 1-11
Advanced Telecommunications Computing
Architecture (ATCA), 1-1
ARTM, 1-11, 2-18, 5-4, 5-10
installing, 2-19
ATCA (AdvancedTCA), 1-1
ATCA midplane, 2-20
auto-boot (OpenBoot variable), 4-6
automatic system recovery (ASR)
disabling, 4-17
enabling, 4-17
obtaining recovery information, 4-28
replacing, 2-13
compliance
NEBS, 1-3
PICMG specifications, 1-3
configurations
Ethernet, 1-13
I/O, 1-13
Netra switches, 3-9
SAS, 1-13
connectors
pinouts, A-3 to A-13
console port, 5-4, 5-11
cores, processor, 5-2
B
D
A
base interface, 5-3, 5-9, 5-10
battery
TOD clock, 2-14
blade server
installation, 2-1, 2-21
layout, A-1
locking screws, 2-22
block diagram, 5-2
C
chip multithreading (CMT), 5-4
CMT (chip multithreading), 5-4
Compact Flash, 1-8, 1-13, 1-14, 2-5, 2-13, 5-12
adding, 2-13
formatting, 3-10
location, 2-14
device
information, retrieving, 4-19
device paths, 4-8
diagnostics
POST, 4-3
SunVTS software, 3-9
diskless clients, 3-4
adding, 3-5
boot server, 3-4
Dual Star topology, 5-1
E
electric shock caution, 2-1
Electronic Fuse (Efuse), 5-7
Ethernet
10-Gb XAUI, 5-1
base interface, 5-1
Index-1
fabric interface, 5-1
management port, 5-4, 5-11
RJ-45, A-4
SERDES, 5-1
F
fabric interface, 5-1, 5-3, 5-9, 5-10
fan tray upgrade kit, 1-14
FB-DIMMs, 2-5, 5-8
description, 2-5
installing, 2-11
location, 2-6
removing, 2-10
requirements, 2-5
features, 1-2
CPU, 1-2
hot-swap, 1-3
IPMI system management, 1-3
memory, 1-2
operating system, 1-3
power requirements, 1-2
Field-Programmable Gate Array (FPGA), 5-12
firmware updates, 3-8
flash update, 1-3
form factor, A-1
front panel, A-3, A-5
I/O, 1-3, 5-11
FRUID PROMs, 5-17
frutree
entries and properties, 4-19
hierarchy, 4-19
G
graceful shutdown, 4-5
H
hardware
architecture, 5-1
descriptions, 5-1, 5-18
device paths, 4-8
host adapter (probe-scsi), 4-7
hot-swap
LED, 2-25
support, 1-14
hot-swap latch, 2-25
Index-2
I
I/O, 5-3
dual USB ports, 5-3
front panel, 1-3, 5-11
IDE port (Compact Flash), 5-3
resets, 5-17
I2C interface, 5-18
IDE bus, 4-8
init (Solaris command), 4-5
injector/ejector latches, 2-21
install patches, 3-5
installation
collecting network information, 3-7
FB-DIMMs, 2-5, 2-11
overall procedure, 2-4
preparation, 2-3
software, 3-1
summary, 2-4
tools needed, 2-3
installing blade server, 2-22
integrated drive electronics (IDE), 4-8
Intelligent Platform Management Bus (IPMB), 5-14
Intelligent Platform Management Controller
(IPMC), 4-2
Internet Assigned Numbers Authority (IANA), B-1
IPMB-L, 5-14
IPMC, 4-2, 5-13
IPMI system management, 1-3
L
L2 cache, 5-8
labels, 1-16
MAC address, 1-16
part number, 1-16
serial number, 1-16
latch
hot-swap, 2-25
LEDs
active, A-3
fault, 5-15, A-3
front panel, A-3
healthy status, 5-15
hot-swap, 2-25, 5-15, A-3
local network information, 3-7
logical unit number (probe-scsi), 4-7
loop IDs (probe-scsi), 4-7
Netra CP3260 Blade Server User’s Guide • April 2009
M
MAC address, 3-8
label, 1-16
management port
Ethernet, 5-11
manual system reset, 4-5
memory, 1-2, 5-8
FB-DIMMs, 2-5, 5-9
N
netfunction (NetFn), B-1
Netra CP32X0 ARTM, 1-8, 1-11, 1-13, 1-14, 2-20
network boot, 1-8
network device aliases, 4-18
Power PC, 5-3
power-on self-test (POST), 4-2
preparation
installation, 2-3
probe-ide (OpenBoot command), 4-8
probe-scsi (OpenBoot command), 4-7
probe-scsi-all (OpenBoot command), 4-7
processor
block diagram, 5-5
cores, 5-2, 5-4
speed, 5-4
UltraSPARC T2, 5-2, 5-4
R
OpenBoot
ok prompt, 4-5
OpenBoot commands
probe-ide, 4-8
probe-scsi, 4-7
probe-scsi-all, 4-7
show-devs, 4-8
watch-net, 4-11
OpenBoot variables
auto-boot, 4-6
displaying, 4-14
setting, 4-14
operating system, 3-1
output power, 5-18
rear transition module, 1-11
removable media device, 3-10
removing FB-DIMMs, 2-10
requirements
hardware, 1-14
operating system, 1-3
safety, 2-1
software, 1-15
reset
manual system, 4-5
revision number, 1-16
RJ-45 Ethernet, A-4
rmformat utility, 3-10
RTM, 2-18
compatibility, 2-18
support, 5-4
P
S
part number, 1-16
patches, 3-1
PCI Express switch, 5-10
PICL, 4-19
PICL (platform information and control library), 419
pinouts
connectors, A-3 to A-13
POST
error messages, 4-3, 4-4
tests, 4-3
POST (power-on self test), 4-3
power module, 5-18
SCSI devices
diagnosing problems, 4-7
SERDES, 5-1, 5-4, 5-10
serial number, 1-16
serial port, 5-11, A-5
service processor, 5-12
shielded cables, 1-13, 2-24
show-devs (OpenBoot command), 4-8
shutdown (Solaris command), 4-5
software
patches, 3-1
PICL, 4-19
Solaris OS, 3-1
O
Index-3
Solaris commands
init, 4-5
shutdown, 4-5
SPARC, 5-2
subsystems, 5-1
CPU and memory, 5-1
I/O, 5-1
IPMI, 5-1
power, 5-1
service processor, 5-1
Sun Download Center (SDLC), 3-1
Sun OEM IPMI commands
Get RTM Status, B-3
Sun Services, 1-15
SunVTS software, 3-9
support, 1-15
switch configuration
Netra CP3140 1-GbE, 3-8
Netra CP3240 10-GbE, 3-8
system flash PROM, 4-2
system monitor, 5-15
World Wide Name (probe-scsi), 4-7
X
XAUI, 5-1, 5-8, 5-9
XAUI ports, 5-8
T
temperature monitoring, 5-16
TOD clock
battery, 5-18, A-13
battery holder, A-13
TOD clock battery
adding, 2-14
replacing, 2-15
tools needed, installation, 2-3
topology
Dual Star, 5-1
U
UltraSPARC T2, 1-1, 1-2, 1-8, 5-2, 5-4, 5-10, 5-12, 5-18
upgrade kit
fan tray, 1-14
USB port, 5-12
V
voltage monitoring, 5-15
voltage sensor thresholds, 5-16
W
warranty, 1-15
Index-4
Netra CP3260 Blade Server User’s Guide • April 2009