Sun Datacenter Switch 3456 Site Planning Guide

Sun™ Datacenter Switch 3456 Site
Planning Guide
Sun Microsystems, Inc.
www.sun.com
Part No. 820-4728-10
August 2008, Revision A
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Copyright 2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In
particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at
http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and in other
countries.Unpublished - rights reserved under the Copyright Laws of the United States..
This distribution may include materials developed by third parties.Parts of the product may be derived from Berkeley BSD systems, licensed
from the University of California. UNIX is a registered trademark in the U.S. and in other countries, exclusively licensed through X/Open
Company, Ltd.
Sun, Sun Microsystems, the Sun logo, SunService, and The Network is the Computer are trademarks or registered trademarks of Sun
Microsystems, Inc. in the U.S. and other countries.The OPEN LOOK and Sun(TM) Graphical User Interface was developed by Sun
Microsystems, Inc. for its users and licensees.
Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the
computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s
licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.
Use of any spare or replacement CPUs is limited to repair or one-for-one replacement of CPUs in products exported in compliance with U.S.
export laws. Use of CPUs as product upgrades unless authorized by the U.S. Government is strictly prohibited.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright © 2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.
Sun Microsystems, Inc. détient les droits de propriété intellectuels relatifs à la technologie incorporée dans le produit qui est décrit dans ce
document. En particulier, et ce sans limitation, ces droits de propriété intellectuelle peuvent inclure un ou plus des brevets américains listés à
l’adresse http://www.sun.com/patent set un ou les brevets supplémentaires ou les applications de brevet en attente aux Etats - Unis et
dans les autres pays.Non publie - droits réservés selon la législation des Etats-Unis sur le droit d’auteur.
Cette distribution peut comprendre des composants développés par des tierces parties.Des parties de ce produit pourront être dérivées des
systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque déposée aux Etats-Unis et dans d’autres pays et licenciée
exclusivement par X/Open Company, Ltd.
Sun, Sun Microsystems, le logo Sun, SunService, et The Network is the Computer sont des marques de fabrique ou des marques déposées de
Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.L’interface d’utilisation graphique OPEN LOOK et Sun(TM) a été développée par
Sun Microsystems, Inc. pour ses utilisateurs et licenciés.
Sun reconnaît les efforts de pionniers de Xerox pour la recherche et le développement du concept des interfaces d’utilisation visuelle ou
graphique pour l’industrie de l’informatique. Sun détient une license non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette
licence couvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui, en outre, se
conforment aux licences écrites de Sun.
L’utilisation de pieces detachees ou d’unites centrales de remplacement est limitee aux reparations ou a l’echange standard d’unites centrales
pour les produits exportes, conformement a la legislation americaine en matiere d’exportation. Sauf autorisation par les autorites des EtatsUnis, l’utilisation d’unites centrales pour proceder a des mises a jour de produits est rigoureusement interdite.
LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFACON.
Please
Recycle
Contents
Preface
1.
vii
Cautions and Considerations
Chassis Weight
1
1
Moving the Chassis
1
Loading by the Chassis
Chassis Activity Dimensions
Heat Considerations
3
Air Temperature
3
Surface Temperature
2
3
4
Other Objects’ Temperatures
Hot and Cold Aisles
4
Electrical Considerations
4
Main Power
2.
4
4
Bus Power
5
Pin Power
5
Sun Datacenter Switch 3456 Specifications
Weight
7
7
Chassis Weight
7
iii
Component Weights
External Dimensions
8
8
Chassis External Dimensions
Component External Dimensions
9
Shipping Container Dimensions
10
Environmental Specifications
Power Consumption
3.
9
10
11
Partial Configuration
13
Complete and Base Configurations
Configuration Considerations
Filler Panels
14
14
Line Card Distribution
Power Supplies
4.
14
14
Component Distribution
15
Location Requirements
17
Equipment to Unload and Move
Forklift
18
Other Equipment
18
Path to Final Location
18
Security Systems
Signage
19
Carpets
19
Grades
19
Elevators
18
20
20
Thresholds and Gaps
iv
17
17
Pallet Jack
Doors
13
20
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Path
20
Surface
20
Turns
21
Final Location
21
Floor Design and Loading
Working Space
Administration
22
22
Dedicated Servers
Interfaces
21
22
23
Switch-Specific Interface
CLIA Interface
23
Linux Interface
23
Pin Repair Kit
5.
23
23
Environmental Requirements
Heat and Airflow
25
25
Heat Generated That Has to Be Replaced
Air Flow and Thermal Channels
Near Stationary Objects
Near Other Heat Sources
Example Situation
Electric Power
25
26
26
27
28
28
Power Delivery
28
Power Backup
Power Cables
28
29
Cable Conduit
Circuit Breakers
Local Codes
29
29
29
Contents
v
6.
InfiniBand Cabling Requirements
IB Cable Cautions
31
31
IB Cable Routing
32
Bundling IB Cables
32
Floor and Underfloor Delivery of IB Cables
Overhead Delivery of IB Cables
IB Cable Length
32
32
33
Cable Type and Length
33
Determining IB Cable Path Length at Sun Datacenter Switch 3456
Cable Connection
7.
36
Personnel Requirements
Installation Personnel
37
37
Installation Responsibilities
37
Installer Education and Experience
Administration Personnel
38
38
Administration Responsibilities
38
Administrator Education and Experience
Maintenance and Service Personnel
39
39
Maintenance and Service Responsibilities
39
Maintenance and Service Education and Experience
8.
Site Planning Checklist
Thermal Checklist
Power Checklist
41
41
Network Checklist
42
Location Checklist
42
Personnel Checklist
Index
vi
41
43
45
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
39
34
Preface
This site planning guide provides information about the Sun Datacenter Switch 3456
switch and the facilities to accommodate the switch.
This document is written for persons responsible for organizing the installation of
the Sun Datacenter Switch 3456, and configuring its working environment. Topics
include cautions, specifications, location, utilities, and personnel.
Related Documentation
The documents listed as online are available at:
http://docs.sun.com/app/docs/prod/switch.3456
Application
Title
Part Number
Format
Location
Product Notes
Sun Datacenter Switch 3456 Product Notes
820-4727-10
PDF
Online
Unpacking
Sun Datacenter Switch 3456 Unpacking Guide
820-4736-10
PDF
Printed
Shipping crate
Online
Site Planning
Sun Datacenter Switch 3456 Site Planning Guide
820-4728-10
PDF
Online
Installation
Sun Datacenter Switch 3456 Installation Guide
820-4730-10
PDF
Printed
Shipping kit
Online
Administration Sun Datacenter Switch 3456 Administration Guide
820-4731-10
PDF
Online
Service
Sun Datacenter Switch 3456 Service Manual
820-4733-10
PDF
Online
Reference
Sun Datacenter Switch 3456 Reference Manual
820-4734-10
PDF
Online
Regulatory
Sun Datacenter Switch 3456 Safety and Compliance
Guide
820-4735-10
PDF
Online
vii
Documentation, Support, and Training
Sun Function
URL
Documentation
http://www.sun.com/documentation/
Support
http://www.sun.com/support/
Training
http://www.sun.com/training/
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun Datacenter Switch 3456 Site Planning Guide, part number 820-4728-10.
viii
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
1
Cautions and Considerations
This chapter describes the challenges in preparing the site and installing the Sun
Datacenter Switch 3456. This chapter contains the following topics:
■
“Chassis Weight” on page 1
■
“Chassis Activity Dimensions” on page 3
■
“Heat Considerations” on page 3
■
“Electrical Considerations” on page 4
Note – A hard copy version of the Sun Datacenter Switch 3456 Installation Guide is
shipped inside the switch shipping crate. You can also access the guide at this URL:
http://docs.sun.com/app/docs/prod/switch.3456
Chassis Weight
As shipped, the Sun Datacenter Switch 3456 switch chassis and crate weighs
approximately 1800 pounds (820 kg). This section describes the effects and behavior
of that much mass.
Moving the Chassis
When moving the chassis from the truck to the unpacking area and to the final
location, consider the implications of the chassis weight:
■
Traversing any grade requires additional force and personnel to safely move the
Sun Datacenter Switch 3456.
1
■
The Sun Datacenter Switch 3456 casters all swivel freely. The chassis responds to
the greater of applied forces. This means that a person pushing on one corner to
move the chassis also causes the chassis to rotate. So at least two people should
push equally on adjacent corners to move the Sun Datacenter Switch 3456.
■
When rolling the Sun Datacenter Switch 3456 off of the shipping pallet and down
the shipping ramp, the chassis catches at the junction of the pallet and ramp, and
has a tendency to rotate. Expect and prepare for this behavior.
■
To accelerate the Sun Datacenter Switch 3456 to a walking speed requires effort,
time, and distance. To stop the Sun Datacenter Switch 3456 from moving requires
the same or more. To accelerate, transport, or stop a Sun Datacenter Switch 3456
requires slow methodical movement, a clear and safe path, and sufficient
personnel aware of their actions.
Caution – The Sun Datacenter Switch 3456 chassis cannot be safely moved by one
person alone. Never allow any person near the direct path of the Sun Datacenter
Switch 3456. Personnel moving the Sun Datacenter Switch 3456 should consciously
keep their feet away from the bottom edge of the Sun Datacenter Switch 3456,
because this is a pinch point and can cause injury. Personnel moving the Sun
Datacenter Switch 3456 should stand to either side of the chassis or behind its
direction of motion.
Loading by the Chassis
The Sun Datacenter Switch 3456 can only be located on surfaces that can
accommodate its weight. The four casters on the Sun Datacenter Switch 3456 chassis
have a total contact area of 2 square inches or a loading of 726 psi. This load should
not be of concern, if the surfaces or foundations are concrete or reinforced.
Unreinforced wood surfaces cannot support the weight of the Sun Datacenter Switch
3456, and even if only temporary during movement, might flex enough to cause
bottoming out.
Carpeted surfaces might not be able to survive the stress of the Sun Datacenter
Switch 3456 casters, especially when turning. Additionally, movement over carpet is
difficult.
2
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Chassis Activity Dimensions
The following table provides Sun Datacenter Switch 3456 chassis dimensions, and
the minimum dimensions needed for unpacking, moving, and the final location.
TABLE 1-1
Approximate Minimum Chassis Activity Dimensions
Item
Length
Width
Height
Comment
Sun Datacenter
Switch 3456
chassis
6 ft (1.8 m)
4 ft (1.2 m)
5 ft (1.5 m)
Dimensions of chassis.
Unpacking space
35 ft (10.7 m)
10 ft (3 m)
9 ft (2.7 m)
To accommodate rollout
and crate shell movement
Moving perimeter 15 ft (4.6 m)
10 ft (3 m)
6 ft (1.8 m)
For stopping distance and
safety margin.
Installation at
final location
12 ft (3.7 m)
12 ft (3.7 m)
6 ft (1.8 m)
To accommodate line card
and fabric card
installation.
Final location
8 ft (2.4 m)
10 ft (3 m)
6 ft (1.8 m)
Dimensions outside of
cable guides, access space
only.
Heat Considerations
This section describes the heat output of the Sun Datacenter Switch 3456 and its
potential safety hazard for personnel. This section also describes keeping an open
area at the exhaust side, and considerations for other systems and equipment in the
area.
Air Temperature
The calculated approximate air temperature output of the Sun Datacenter Switch
3456 is 36°F (20°C) hotter than the air going in. While this output temperature is not
hot enough to burn, this temperature is sufficient to dehydrate and render useless
any cooling device in the vicinity.
Chapter 1
Cautions and Considerations
3
Surface Temperature
During operations, the exhaust surfaces, such as the fans mounted to the fabric
cards, are at a higher temperature than the exhaust air. Bear this situation in mind
when replacing a failed fan, because its temperature can be even greater.
Other Objects’ Temperatures
No object should be in direct contact with the exhaust air. However, occasional
hanging cables or drop lamps might happen to be in the stream. Any objects in the
exhaust air stream is at an elevated temperature because of this exposure.
Hot and Cold Aisles
The heat output of a fully configured Sun Datacenter Switch 3456 is approximately
40 KW. The Sun Datacenter Switch 3456s’s air mass flow is calculated to
approximately 3400 cfm (1.6 m3/s), which means that the air exhausted from the
switch is 36°F (20°C) degrees hotter than the air going in. As such, the hot aisle for
the Sun Datacenter Switch 3456 should be configured to accommodate this heat
output and in no way compromise the cooling of other equipment. Do not store any
objects in the switch hot aisle. Such objects would interfere with proper air
circulation, and might become a safety hazard.
Electrical Considerations
This section describes the voltages present and the exposure to active lines during
component installation.
Main Power
Caution – The 16 Sun Datacenter Switch 3456 power supplies require 200 -240 VAC.
Take all precautions normally associated with these voltages.
4
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Bus Power
Caution – When servicing the Sun Datacenter Switch 3456, bus power is active. Bus
power is a low voltage, but very high current. Even the smallest form of short circuit
can cause physical injury and severe damage to the Sun Datacenter Switch 3456.
Pin Power
Caution – When servicing the Sun Datacenter Switch 3456, pin power and signals
are active. While not a shock or burn threat, there is the possibility of damage to the
pin, midplane, fabric card, and line card, should a short or grounding take place.
Use all precautions associated with working with active signal conductors when
servicing the Sun Datacenter Switch 3456.
Chapter 1
Cautions and Considerations
5
6
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
2
Sun Datacenter Switch 3456
Specifications
This chapter describes specifications of the Sun Datacenter Switch 3456 chassis,
components, and shipping containers. Topics include:
■
“Weight” on page 7
■
“External Dimensions” on page 8
■
“Environmental Specifications” on page 10
■
“Power Consumption” on page 11
Weight
This section describes minimum and maximum weights of the Sun Datacenter
Switch 3456 chassis, components, and the shipping containers.
Chassis Weight
TABLE 2-1 lists the minimum and maximum chassis weights.
TABLE 2-1
Chassis Weight According to Configuration
Component
Weight
Bare chassis
992 lb (450 kg)
Chassis as shipped
1433 lb (650 kg)
7
TABLE 2-1
Chassis Weight According to Configuration (Continued) (Continued)
Component
Weight
Chassis with 18 fabric cards and 1 line card
2128 lb (965 kg)
Chassis fully configured
3043 lb (1380 kg)
Chassis fully configured with cabling and hardware
4325 lb (1966 kg) approximated
Component Weights
TABLE 2-2 provides the weight of line cards, fabric cards, Chassis Management
Controllers (CMC)s, and power supplies.
TABLE 2-2
Weight of Chassis Components
Component
Weight
Fan module
< 1 lb (0.5 kg)
CMC
< 1 lb (0.5 kg)
Power supply
4.4 lb (2 kg)
Fabric card
36.4 lb (16.5 kg)
Line card
39.7 lb (18 kg)
Splitter cable
0.104 lb/ft (0.155 kg/ m)
Cable tree
66.9 lb(30.4 kg)
Cable tray
7.3 lb (3.3 kg)
Cable plate
3.3 lb (1.5 kg)
External Dimensions
This section describes external dimensions.
8
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Chassis External Dimensions
TABLE 2-3 lists the dimensions of the configured chassis, and of the working space
around the chassis.
TABLE 2-3
Chassis Dimensions and Working Space
Dimension
Chassis
Working Space
Total
Length (left right)
71.8 in (1.823 m)
22 in (0.56 m)(left)
22 in (0.56 m)(right)
115.8 in (2.94 m)
Width (front rear)
43.8 in (1.113 m)
44 in (1.12 m)(front)
44 in (1.12 m)(rear)
131.8 in (3.35 m)
Height
62.4 in (1.585 m)
12 in (0.3 m)
74.4 in (1.89 m)
Component External Dimensions
TABLE 2-4 provides the approximate dimensions of the chassis components.
TABLE 2-4
Approximate Component Dimensions
Component
Length
Width
Height
Fan module
5.3 in (135 mm)
1.6 in (42 mm)
4.3 in (110 mm)
CMC
14 in (357 mm)
13.9 in (355 mm)
1.2 in (30 mm)
Power supply
10 in (256 mm)
4.2 in (106 mm)
2.8 in (71 mm)
Fabric card
45.5 in (1156 mm)
14.8 in (375 mm)
1.8 in (47 mm)
Line card
42.8 in (1088 mm)
13.3 in (337 mm)
1.6 in (41 mm)
Cable tree
18.9 in (480 mm)
13.3 in (337 mm)
41.3 in (1050 mm)
Cable tray
12.6 in (320 mm)
48.5 in (1233 mm)
0.6 in (15 mm)
Cable plate
8.3 in (211 mm)
9.7 in (246 mm)
2 in (52 mm)
Chapter 2
Sun Datacenter Switch 3456 Specifications
9
Shipping Container Dimensions
TABLE 2-5 lists dimensions of the shipping containers and notes regarding storing the
containers.
TABLE 2-5
Shipping Container Dimensions and Notes
Shipping Container
Component
Dimensions of Container
Notes for Storage
Chassis
90 in x 40 in x 60 in
(2.3 m x 1 m x 1.5 m)
Store as a closed container. Internal
space can accommodate many fabric
card and line card boxes.
Fabric card and
line card
53.7 in x 23.4 in x 5.1 in
(1365 mm x 595 mm x 130 mm)
Empty fabric card and line card boxes
can be stacked inside of the chassis
shipping crate.
Environmental Specifications
TABLE 2-6 lists environmental specifications for operation and nonoperation of the
Sun Datacenter Switch 3456.
10
TABLE 2-6
Environmental Specifications
Parameter
Operating Value
Nonoperating Value
Temperature
5 – 35 ˚C
-40 – 65˚C
Humidity
10% RH @ 27˚C wet bulb
90% RH @ 27˚C
Altitude
3000 m @ 35˚C
12,000 m
Shock
3 G, 11 msec, 1/2 sine
1 in roll-off, 20 mm @ 0.75 m/s
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Power Consumption
The Sun Datacenter Switch 3456 has 16 power supplies. TABLE 2-7 provides power
supply requirements for supplied power:
TABLE 2-7
Individual Power Supply Power Requirements
Aspect
Value
Voltage
200 – 240 VAC
Frequency
50 – 60 Hz
Current
13 Amps @ 200 VAC nominal, 14.8 Amps @ 200 VAC maximum
TABLE 2-8 provides a breakdown of the power requirements for the Sun Datacenter
Switch 3456 chassis components.
TABLE 2-8
Chassis Component Power Requirements
Component
Quantity
Standby (3.3 V)
Power (each)
Operational (12V)
Power (each)
Total for Components
CMC
2
6.27
0
13.54
Fans
144
0
20.04
2885.76
Fabric card
18
2.97
370.44
6721.38
Line card
24
2.97
1235.16
29715.12
138.28
39197.52
39335.80
Total
The total power consumed is also the heat load on the environmental control
systems by the Sun Datacenter Switch 3456.
Chapter 2
Sun Datacenter Switch 3456 Specifications
11
12
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
3
Partial Configuration
This chapter describes configuring the Sun Datacenter Switch 3456 with less than a
complete configuration. Topics include:
■
“Complete and Base Configurations” on page 13
■
“Configuration Considerations” on page 14
■
“Component Distribution” on page 15
Complete and Base Configurations
For maximum functionality, the Sun Datacenter Switch 3456 is designed to be
operated in a fully configured state:
■
24 line cards
■
18 fabric cards
■
16 power supplies
■
2 CMCs
If necessary, fewer components can be installed. For example, the base configuration
of the switch is:
■
1 line card
■
18 fabric cards
■
6 power supplies
■
2 CMCs
For the base configuration or a partial configuration, filler panels must be installed
where there are empty line card and power supply slots.
13
Caution – Do not configure the Sun Datacenter Switch 3456 with fewer than 18
fabric cards installed. Thermal requirements will not be satisfied.
Configuration Considerations
Filler Panels
To maintain proper airflow for adequate cooling, filler panels must be installed in
vacant line card and power supply slots. Besides directing airflow, the filler panels
close openings to the Sun Datacenter Switch 3456 chassis, preventing intrusion of
objects and contaminants to the chassis interior.
Line Card Distribution
Line cards are installed from the center out, across the available slots. The vacant
slots must have filler panels installed. For example, if only 12 line cards are to be
installed, they would occupy slots 6 through 17. Filler panels are installed in the
vacant slots (0 through 5 and 18 through 23).
Power Supplies
Power supplies are configured in a need+1 redundancy. A fully configured Sun
Datacenter Switch 3456 with 24 line cards and 18 fabric cards needs 14 power
supplies. Two power supplies act as redundant, one for each power bus. The power
supply slots are hard-wired to two power buses. Power supplies installed in the
lower row (PS0, PS1, and so on to PS7), provide power to line card slots 0 to 11 and
fabric card slots 9 to 17. Power supplies installed in the upper row (PS8, PS9, and so
on to PS15), provide power to line card slots 12 to 23 and fabric card slots 0 to 8.
14
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
TABLE 3-1 lists the maximum number of line cards that are supported for a given
number of power supplies.
Maximum Quantity of Line Cards for Quantity of Power Supplies
TABLE 3-1
Quantity of Power Supplies
Maximum Supported Line Cards
6
2
8
6
10
10
12
16
14
20
16
24
Component Distribution
If you are configuring less than a complete Sun Datacenter Switch 3456, you can
incrementally add and distribute line cards and power supplies to the empty slots in
the left-to-right order described by TABLE 3-2:
TABLE 3-2
Incremental Configuration
Quantity of Line Cards
Line
Card Slot
Power
Supply
Slots
1
2
11
12
2, 4, 6, 9,
11, 13
3
4
10 13
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
9
14
8
15
7
16
6
17
5
18
4
19
3
20
2
21
1
22
0
23
0, 15
7, 8
3, 12
1, 14
5, 10
For example, if you were to configure the switch with only one line card, you would
install the line card into slot 11. You would also install power supplies into slots 2, 4,
6, 9, 11, and 13, in that order.
If you were to configure the switch with ten line cards, you would install the line
cards into slots 11, 12, 10, 13, 9, 14, 8, 15, 7, and 16, in that order. You would also
install power supplies into slots 2, 4, 6, 9, 11, 13, 0, 15, 7, and 8, in that order.
Chapter 3
Partial Configuration
15
TABLE 3-3 provides suggested configurations, based upon a fraction of the complete
configuration.
TABLE 3-3
Suggested Configurations
Configuratio
n
Quantity
of Line
Cards
Line Card Slots
Power Supply Slots
Base
1
11
2, 4, 6, 9, 11, 13
1/8
3
11, 12, 10
2, 4, 6, 9, 11, 13, 0, 15
1/4
6
11, 12, 10, 13, 9, 14
2, 4, 6, 9, 11, 13, 0, 15
3/8
9
11, 12, 10, 13, 9, 14, 8, 15, 7
2, 4, 6, 9, 11, 13, 0, 15, 7, 8
1/2
12
11, 12, 10, 13, 9, 14, 8, 15, 7, 16, 6, 17
2, 4, 6, 9, 11, 13, 0, 15, 7, 8, 3, 12
5/8
15
11, 12, 10, 13, 9, 14, 8, 15, 7, 16, 6, 17, 5, 18, 4
2, 4, 6, 9, 11, 13, 0, 15, 7, 8, 3, 12
3/4
18
11, 12, 10, 13, 9, 14, 8, 15, 7, 16, 6, 17, 5, 18, 4, 19,
3, 20
2, 4, 6, 9, 11, 13, 0, 15, 7, 8, 3, 12, 1, 14
7/8
21
11, 12, 10, 13, 9, 14, 8, 15, 7, 16, 6, 17, 5, 18, 4, 19,
3, 20, 2, 21, 1
2, 4, 6, 9, 11, 13, 0, 15, 7, 8, 3, 12, 1, 14,
5, 10
Because the line cards are equally distributed across the available slots, cabling rules
and lengths still apply. See Chapter 6.
16
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
4
Location Requirements
This chapter discuses the location equipment, spatial, and loading requirements and
other requirements. Topics include:
■
“Equipment to Unload and Move” on page 17
■
“Path to Final Location” on page 18
■
“Final Location” on page 21
■
“Administration” on page 22
■
“Pin Repair Kit” on page 23
Equipment to Unload and Move
This section describes the tools to unload the Sun Datacenter Switch 3456 from the
truck, and move it to the final location. Ratings of the forklifts, pallet jacks, and other
equipment.
Forklift
If a forklift is used for moving the shipping crates, it must be rated for at least 2000
lbs. (910 kg) lift. The forks must extend at least 4 ft (1.3 m), the width of the chassis
shipping crate.
Follow these guidelines when using a forklift for moving the shipping crates:
■
Stack fabric card or line card shipping crates no more than four high for
movement.
■
When moving a crate, raise it only high enough to enable movement or clear any
floor obstructions (thresholds).
17
Caution – Do not use a forklift for moving the Sun Datacenter Switch 3456 chassis.
There is not enough clearance for the forks under the chassis. Lifting from any
location on the chassis is not possible and can damage the chassis.
Pallet Jack
If a pallet jack is used for moving the shipping crates it must be rated for at least
2000 lbs. (910 kg) lift. Stack fabric card or line card shipping crates no more than four
high for movement.
Other Equipment
Do not use the following for moving crates:
■
Server lift
■
Hand truck
■
Hand cart
Note – These devices can be used for moving cardboard shipping boxes, however.
Path to Final Location
This section address obstructions or hindrances along the route from the unpacking
site to the final location.
Security Systems
The path to the final location might need the Sun Datacenter Switch 3456 to move
through a secure area. Sensors for intrusion, movement, doors opening, and so on,
must be disabled, or else a false alarm might sound.
Consider the impact of the path to the final location on the security of the building.
Are the personnel entering or passing through an area that they are not authorized
to do so?
18
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Consult with the security services before moving the Sun Datacenter Switch 3456.
This way, security is not surprised if intrusion and door alarms are activated.
Additionally, security might be able to provide an alternative route that has less
impact to building security.
Signage
If the path to the final location has high foot traffic, consider posting signs and
providing email informing personnel of the move. This action alerts uninvolved
personnel to avoid the path, which eases the movement of the Sun Datacenter
Switch 3456 and reduces the possibility of complications or injuries.
Carpets
The Sun Datacenter Switch 3456 chassis cannot roll on carpet without great effort.
Plan a path to the final location that avoids carpeted areas. If carpeting cannot be
avoided, place sheet metal or masonite panels over the carpeting to make the chassis
roll more easily.
You do not need to place panels on the entire carpeted path. Use enough to support
the chassis and provide the chassis a surface to roll on. For example, if you use 3 ft x
5 ft (0.91 m x 1.52 m) masonite panels, three panels support the chassis, while a
fourth panel can be laid in front of the chassis in the direction of travel. Once the
chassis has rolled off the rear-most panel, lift that panel and move it to the front.
Repeat this process down the path over carpeting.
Grades
The Sun Datacenter Switch 3456 chassis weighs 1433 pounds (650 kg) as shipped.
Any grade along the path requires additional effort to move the chassis, or prevent
the chassis from moving. Do not attempt to move the chassis up or down a grade
greater than 3%.
Additionally, the Sun Datacenter Switch 3456 chassis is equipped with antitilt bars
that are mounted at the centerline. The presence of the bars limits the clearance
movement to 0.5 inch (12 mm). When a chassis peaks a grade of 3% or greater, the
antitilt bars bottom out.
Chapter 4
Location Requirements
19
Elevators
If the path to the final location requires use of an elevator, ensure that the elevator
can safely accommodate the weight of the Sun Datacenter Switch 3456 chassis and
the personnel moving it.
The elevator doors must be wide enough to allow the Sun Datacenter Switch 3456 to
pass, ideally with personnel alongside. The doors might need to be held open, either
physically, or by pressing the door open button within the elevator.
Verify that the threshold at the elevator doors permits the chassis to enter and exit
the elevator. The gap between the floor and elevator should not be too large, nor
should there be any height difference. Check this gap at both the entering and
exiting floors. If there are any minor discrepancies, a metal sheet or masonite panel
can be used to compensate.
Doors
Doors must open wide enough to accommodate the Sun Datacenter Switch 3456
chassis and personnel alongside. Additional personnel can hold doors open, or the
doors can be securely propped open using wedges or catches. Do not hold doors
open with chairs or other easily available objects. Such objects do not guarantee a
secure door and can diminish the space for moving the chassis.
Thresholds and Gaps
The path to the final location might have building expansion joint gaps, carpet
runners, door thresholds, or other inconsistencies in an otherwise smooth surface.
When these obstructions cannot be avoided, use metal plates or masonite panels to
compensate for the irregularities. Should a gap be too great, or a threshold too high,
the panel might break. In this situation, an alternative path to the final location must
be found.
Path
Surface
The path, regardless of surface, should be clean and free of any obstructions.
Obstructions include trash, cables, fasteners (screws, nuts, paper clips, thumbtacks,
tie wraps, and so on), any scrap (pencils, pens, papers, and so on), or small object
20
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
that might hinder movement. A quick sweeping of the path to the final location
before moving the Sun Datacenter Switch 3456 can remove most objects that might
later prove troublesome.
Turns
Ensure that any turns in the path to the final location allow unhindered movement
of the chassis. Avoid narrow hallways and tight turns, because they create
opportunities for injury.
Check the path before moving the Sun Datacenter Switch 3456. If you are in doubt of
the chassis’ ability to navigate a certain section of a building, a make-shift
framework, constructed of duct-taped sticks for example, can give an idea of the
path’s viability. Walk the framework through the path to the final location. If there is
any situation where the framework binds, then the chassis will also.
If a tight turn cannot be avoided, consider adding protection to the walls of the
inside corners of the turn. Additionally, consider adding protection to the corners of
the Sun Datacenter Switch 3456 chassis.
Final Location
This section describes the design and capabilities of the final installation location.
Can the floor take the load, what re-enforcements are needed, and so on.
Floor Design and Loading
A fully configured and cabled Sun Datacenter Switch 3456 weighs in excess of 4325
pounds (1966 kgs). The loading is over an area of 3 x 6 feet (0.9 x 1.8 meters). Yet, the
true stress of the weight is over the four casters and four leveling feet. The casters
have a combined contact area of 2 square inches (13 cm2), as does each of the
leveling feet. The 4325 pound weight is distributed over 10 square inches (65 cm2),
or approximately 432 psi (30.2 kg/cm2). Since the leveling feet are adjacent to the
casters, each corner of the chassis bears 1081 pounds (491 kg). Initially, these two
values seem trivial, but consider that the loading is not equally distributed. If a
caster and leveling foot are located in the center of a raised floor tile, that tile bears
the 1081 pounds (491 kg) load. The floor tile and underlying supports must be able
to accommodate this load.
Chapter 4
Location Requirements
21
Working Space
Once in its final location, the Sun Datacenter Switch 3456 requires a certain amount
of working space for the installation of fabric cards, line cards, power cables, and IB
cables.
When installing or servicing the Sun Datacenter Switch 3456, consider these
conditions to working space:
■
Front – Allow at least 4 feet to enable pin inspection, line card installation, cable
tree and cable tray installation, and IB cable installation.
■
Rear – Allow at least 3 feet to enable pin inspection, fabric card installation, and
power supply cable installation.
■
Sides – Allow at least 3 feet for cable tree and cable plate installation, and for
movement around the chassis.
Administration
This section describes what computing or system resources are allocated to the
administration and management of the Sun Datacenter Switch 3456.
Dedicated Servers
The Sun Datacenter Switch 3456’s design is to network many high performance
computing machines. To communicate with the Sun Datacenter Switch 3456, servers
to interface with the switch are needed.
The Sun Datacenter Switch 3456 uses a mini computer in each CMC to monitor and
administrate components on the IPMB bus. The mini computer is called the shelf
manager, and it has a command interface to send commands to and receive
information from components on the IPMB bus. Communicating with the shelf
manager and issuing commands is done through either the serial management port
or network management port of the CMC. Typically, a terminal server or networked
server is used for this task.
The administration of the routing achieved by the Sun Datacenter Switch 3456 is
done using a subnet manager. This software is run on a server external to the switch.
22
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Interfaces
Switch-Specific Interface
There are switch-specific commands available at the shelf manager prompt. These
commands utilize scripts and CLIA interface commands to simplify some switch
administration activities.
CLIA Interface
The CLIA interface is invoked with the clia command at the shelf manager
prompt.
# clia
cli>
Alternatively, commands for the CLIA interface can be run with the clia command:
# clia subcommand
Switch-specific commands and subcommands for the CLIA interface are discussed
in the Sun Datacenter Switch 3456 Administration Guide and Sun Datacenter Switch
3456 Reference Manual.
Linux Interface
The shelf manager also has a Linux command-line interface and minimal file system.
Pin Repair Kit
In the event that a midplane connector pin is damaged, severely bent, or broken, it
must be replaced. Replacement is accomplished using a pin repair kit, available from
Molex.
Chapter 4
Location Requirements
23
24
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
5
Environmental Requirements
This chapter describes the environmental requirements of the Sun Datacenter Switch
3456. Topics include:
■
“Heat and Airflow” on page 25
■
“Electric Power” on page 28
Heat and Airflow
This section describes the cooling requirements of the Sun Datacenter Switch 3456,
and its airflow.
Heat Generated That Has to Be Replaced
In the maximum configuration, the Sun Datacenter Switch 3456 switch consumes 40
KW. This is the additional heat load to the environmental systems of the room where
the Sun Datacenter Switch 3456 is installed.
Though the airflow through the chassis is great, the resulting thermal output creates
a larger heat density in the locality of the Sun Datacenter Switch 3456. As such, the
air conditioning system must accommodate a more localized heat source and
therefore, might require additional ducting or chillers in the vicinity of the Sun
Datacenter Switch 3456.
If cabling to the Sun Datacenter Switch 3456 is routed overhead, this might
complicate the cooling solution.
25
Air Flow and Thermal Channels
In a fully configured Sun Datacenter Switch 3456, the total air flow through the line
cards and fabric cards is calculated as aproximately 3400 cfm (1.6 m3/s) and the
temperature change of intake air in the cool aisle to the exhaust air in the hot aisle is
36°F (20°C). When contrasted with the output temperature of other systems, if the
hot aisle is too narrow, or the airflow from the opposing systems too low, the Sun
Datacenter Switch 3456 has the ability to heat the opposing systems. It is important
to consider this possibility when determining the location of the Sun Datacenter
Switch 3456 with respect to other systems.
Also consider that the air flow rate through the Sun Datacenter Switch 3456 can
cause a low pressure region in the cool aisle immediately in front of the Sun
Datacenter Switch 3456. If this aisle is narrow in regard to the opposing systems, the
Sun Datacenter Switch 3456’s air flow might reduce the air flow in the opposing
systems, affecting their cooling.
Because the Sun Datacenter Switch 3456 is a central component of a Sun Blade™
6048 Modular System cluster, it follows that its physical location within the cluster is
also central. Use this fact when planning the location and routing of cooling and
return air.
Near Stationary Objects
Basic thermal planning guidelines suggests a hot aisle width of 2 feet for two
opposed systems. The guidelines also state that this situation is identical to a
stationary object, such as a wall, being half that distance. It is not advised to have
such a narrow hot aisle with the Sun Datacenter Switch 3456 switch. Allow at least 3
feet from the exhaust (fan surface) of the Sun Datacenter Switch 3456 to any wall.
Even with this distance to the wall, consider the following:
26
■
The wall will become warm to the touch, possibly as hot as the exhaust surface of
the Sun Datacenter Switch 3456.
■
Any object in the exhaust area, including the wall, experiences accelerated aging
and oxidation.
■
Any personnel entering the exhaust area will feel uncomfortably warm.
Prolonged exposure might cause dehydration.
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Near Other Heat Sources
Most likely, the Sun Datacenter Switch 3456 is the hottest object in the data center.
As discussed previously, it has a large heat density. Consider the heat density of
other systems and objects in the vicinity of the Sun Datacenter Switch 3456 when
determining the cooling configuration.
FIGURE 5-1 describes a simplified thermal mapping of the Sun Datacenter Switch 3456
in the datacenter.
FIGURE 5-1
Simplified Thermal Map
The floorplan shown in FIGURE 5-1 has two switches and 82 racks of servers, denoted
by the letter ‘S’. The black rectangles are rackmounted power backup, conditioning,
and distribution units. The grey rectangles denoted with the letter ‘I’ are
rackmounted I/O controllers and interfaces. To improve thermal characteristics, the
hot aisles are closed so that hot aisle/cold aisle mixing is not possible.
Chapter 5
Environmental Requirements
27
Example Situation
When configuring the hot aisle for the switch and other systems, consider the to wall
dimensions as your separation distance. For example, the Sun Datacenter Switch
3456 requires at least 3 feet spacing between the exhaust fan surface and a wall. If
the opposing system has a similar to wall requirement, such as 1 foot, then the
separation between the switch and the system should be 3 + 1 = 4 feet.
Electric Power
This section describes the power requirements for the Sun Datacenter Switch 3456.
Power Delivery
Each Sun Datacenter Switch 3456 power supply should be shunted through a circuit
breaker for facility power. The cabling from the circuit breaker, through the conduit
to the power receptacles should be capable of providing the following power, and
conform to local codes.
■
200 to 240 VAC
■
15 Amps @ 200 VAC
TABLE 5-1 lists cables available from Sun for the power supplies.
TABLE 5-1
Power Delivery Requirements
Part Number
Cable Description
Comment
180-2005-xx
4 m cable with NEMA L6-20P plug
20 A cable has a twist-lock connector.
180-2004-xx
4 m cable with 16A IEC60309 plug
16 A cable is for European standards.
180-2060-xx
4 m cable with Hubbell C320P6W plug
20 A cable and connector is water-resistant.
180-2061-xx
4 m cable with Hubbell C320P6W plug
16 A cable and connector is water-resistant.
Power Backup
For the Sun Datacenter Switch 3456 to remain operational during a blackout, a
back-up power source must be made available. The power draw of 40 KW means
that battery backup is not possible, and a generator is required.
28
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
The current backup power solution of the data center might be able to accommodate
the additional 40 KW draw. If not or you are uncertain, consult with your facilities or
building manager and seek additional assistance to install more or larger generators
or reconfigure the loads on the existing generators.
Power Cables
Cable Conduit
The Sun Datacenter Switch 3456 has 16 power supplies at its lower backside, Each
power supply has a 200 - 240 volt power connector. The routing of power to the Sun
Datacenter Switch 3456 can be either from above or from below. In either situation,
the power cables from the power supplies must not interfere with access to the back
of the Sun Datacenter Switch 3456 for installation, maintenance, or service. The
cables are best routed to either side of the Sun Datacenter Switch 3456, immediately
exiting the power supply.
Additionally, if power for the Sun Datacenter Switch 3456 is to be delivered from
overhead, such conduits must not restrict or interfere with the airflow out the back
of the Sun Datacenter Switch 3456.
Circuit Breakers
For ease of operation, power to the Sun Datacenter Switch 3456 should be
distributed through independent circuit breakers, one for each power supply. In this
manner, better safety is achieved and minimal impact is received should a power
supply over load or fail.
Local Codes
All power cabling, conduits, and supporting equipment must be installed and
energized according to local code.
Chapter 5
Environmental Requirements
29
30
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
6
InfiniBand Cabling Requirements
This chapter describes requirements for the InfiniBand cabling to the Sun Datacenter
Switch 3456. Topics include:
■
“IB Cable Cautions” on page 31
■
“IB Cable Routing” on page 32
■
“IB Cable Length” on page 33
■
“Cable Connection” on page 36
IB Cable Cautions
Caution – Do not allow any IB cable to bend through a less than 5 inch (127 mm)
radius. Tight bends can break the cable internally.
Caution – Do not use zip ties to bundle or support IB cables. The sharp edges of the
ties can break the cables internally
Caution – Do not allow any IB cable to experience extreme tension. Do not pull on
an IB cable or allow it to drag. Unroll an IB cable for its length. Pulling on an IB
cable can break the cables internally.
Caution – Do not twist an IB cable more than 1 revolution for its entire length.
Twisting an IB cable can break the cable internally.
31
Caution – Do not route IB cables where they might be stepped upon or experience
rolling loads. Such a crushing effect can break the cable internally.
IB Cable Routing
Bundling IB Cables
When bundling IB cables in groups, use hook and loop straps to keep cables
organized. If possible, use color-coordinated straps to help identify cables and their
routing. The IB splitter and 4X cables are fairly thick and heavy for their length.
Consider the retention strength of the hook and loop straps when supporting cables.
Bundle as few cables as reasonably possible. If the IB cables break free of their straps
and fall free, the cables might break internally when they strike the floor or are
jerked from tension.
Floor and Underfloor Delivery of IB Cables
The Sun Datacenter Switch 3456 was designed to accept IB cables from floor or
underfloor delivery. The cable plates on the sides of the switch support the weight of
the IB cables. Each plate supports 96 IB cables in eight rows of 12.
Floor and underfloor delivery limits the tension in the IB cable to the weight of the
cable for the height of the Sun Datacenter Switch 3456 switch.
Overhead Delivery of IB Cables
For overhead delivery, it is suggested that cable trays and lattices be used to support
the IB cables. A fully configured Sun Datacenter Switch 3456 has over 1150 IB cables,
so it is important to distribute the load of the cables over as many supports as
reasonably possible.
The cable plates attached to the sides of the Sun Datacenter Switch 3456 chassis
cannot support cable weights from above. However, they can be used for cable
organization.
32
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
If the overhead delivery has a large drop height, consider using an intermediate
support for the IB cables. Use of the support can limit the tension in the IB cable to
the weight of the cable for the distance between the supports and the Sun Datacenter
Switch 3456.
IB Cable Length
Ideally, cables should be as short as possible. When the length of a cable has been
calculated, select the shortest cable to satisfy the length requirement. When
specifying a cable, consider the following:
■
Bends in the cable path increases the required length of the cable. Rarely does a
cable travel in a straight line from connector to connector. Bends (of a minimum 5
in (127 mm) radius) in the cable are necessary, and with each bend, an increase in
the total length.
■
Bundling increases the required length of the cables. Bundling causes one or more
cables to follow a common path. Yet the bend radius is different in different parts
of the bundle. If the bundle is large, unorganized, and there are many bends, one
cable might experience only the inner radius of bends, while another cable might
experience the outer radius of bends. In this situation, the differences of the
required lengths of the cables is quite substantial.
■
If you are routing the IB cable under floor, consider the height of the raised floor
when calculating cable length.
Cable Type and Length
TABLE 6-1 lists the cables available for the Sun Datacenter Switch 3456, their length,
and data rate.
TABLE 6-1
IB Cables for the Sun Datacenter Switch 3456
Cable Type
Length
Data Rate
Standard Sun IB 12x to 12x
3m
Passive DDR
Standard Sun IB 12x to 12x
5m
Passive DDR
Standard Sun IB 12x to 12x
7m
Active DDR
Standard Sun IB 12x to 12x
11 m
Active DDR
Standard Sun IB 12x to 12x
15 m
Active DDR
Chapter 6
InfiniBand Cabling Requirements
33
IB Cables for the Sun Datacenter Switch 3456 (Continued) (Continued)
TABLE 6-1
Cable Type
Length
Data Rate
Splitter Sun IB 12x to 3 by 4x
8m
Active DDR
Splitter Sun IB 12x to 3 by 4x
12 m
Active DDR
Splitter Sun IB 12x to 3 by 4x
16 m
Active DDR
Determining IB Cable Path Length at Sun
Datacenter Switch 3456
The following tables help you determine the length of cable required to plug into a
line card connector, lay on a cable tray, pass through the cable tree, lay over the cable
plate, and touch the floor. To find this path length, add the lengths of the route
through the cable tree, to and from the cable plates, over the cable plates, and if the
connection is at B, add an additional 0.67 in (17 mm).
Note – The values provided in the following tables are approximated from
calculations of metric measurements taken from an early prototype of the Sun
Datacenter Switch 3456 configured with pass-through IB cables. The lengths are not
exact. You should round-up the values to the next reasonable length, so as to
compensate for changes in path or bundling which are not ideal.
Note – The shaded cells of the tables are values used in the example following the
tables.
Use TABLE 6-2 to find the length of cable path from a connector group through the
cable tree. The values in TABLE 6-2 are the same for all line cards.
TABLE 6-2
Cable Lengths From Connector Group Through Cable Tree
Connector
0, 23
1, 22
2, 21
3, 20
4, 19
5, 18
6, 17
7, 16
8, 15
Length
in (mm)
22.05
(560)
24.17
(614)
26.38
(670)
28.66
(728)
30.98
(787)
33.35
(847)
35.71
(907)
38.11
(968)
40.55 42.99 45.39 47.87
(1030) (1092) (1153) (1216)
34
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
9, 14
10, 13
11, 12
Use TABLE 6-3 to find the length of the cable path from the cable tree to the cable
plate and from the cable plate to the floor.
TABLE 6-3
Cable Lengths From Cable Tree to Cable Plate and Cable Plate to floor
Line cards
0, 1, 2, 3
4, 5, 6, 7
8, 9, 10, 11
12, 13, 14, 15
16, 17, 18, 19
20, 21, 22, 23
Length
in (mm)
23.94
(608)
33.7
(856)
44.25
(1124)
53.86
(1368)
64.29
(1633)
74.37
(1889)
Use TABLE 6-4 to find the length of the cable path over the cable plate.
TABLE 6-4
Cable Lengths Over the Cable Plate
Line cards
0, 4, 8, 12, 16, 20
1, 5, 9, 13, 17, 21
2, 6, 10, 14, 18, 22
3, 7, 11, 15, 19, 23
Length
in (mm)
9.33
(237)
10.71
(272)
12.05
(306)
13.43
(341)
For example, how long is the cable path from connector 1B on line card 17 to the
floor?
24.17 (TABLE 6-2) + 64.29 (TABLE 6-3) + 10.71 (TABLE 6-4) + 0.67 (connector B) = 99.84 in
(2536 mm)
The range of cable path lengths vary from:
■
Shortest, 55.32 in (1405 mm) – Connector 0A or 23A in line card 0
■
Longest, 136.34 in (3463 mm) – Connector 11B or 12B in line card 23
As seen in TABLE 6-1, the shortest cable length is 3 m, yet cable paths can be longer
than 3 m. TABLE 6-5 lists the cable paths greater than 3 m.
TABLE 6-5
Cable Paths Greater than 3 m
Line Card
Connector Groups
LC23
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19
LC22
5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
LC21
5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18
LC20
6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17
LC19
8, 9, 10, 11, 12, 13, 14, 15
LC18
9, 10, 11, 12, 13, 14
LC17
9, 10, 11, 12, 13, 14
LC16
10, 11, 12, 13
Chapter 6
InfiniBand Cabling Requirements
35
Note – Considering the number and range of cable lengths supported by the cable
management hardware, if the cable weighs 0.104 lb/ft (0.155 kg/m), then a fully
configured Sun Datacenter Switch 3456 has an approximate additional weight of
942.7 lbs (428.5 kgs) from just the cables.
Cable Connection
When the Sun Datacenter Switch 3456 is initially installed, cables are attached
starting at the lowest line card and working up to the top line card. This method
keeps an open area above the immediate connection. Once the Sun Datacenter
Switch 3456 is fully configured, removing and attaching cables becomes more
difficult. The free space surrounding an attached cable is fairly tight. Likewise,
connecting a cable in such tight quarters is difficult.
36
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
7
Personnel Requirements
This section describes the people who work with the Sun Datacenter Switch 3456.
Topics include:
■
“Installation Personnel” on page 37
■
“Administration Personnel” on page 38
■
“Maintenance and Service Personnel” on page 39
Installation Personnel
This section describes the personnel who installs the Sun Datacenter Switch 3456.
Installation is a hands-on role.
Installation Responsibilities
The personnel who installs the Sun Datacenter Switch 3456 must be fully capable of
the following tasks:
■
Use of basic hand tools
■
Operation of forklift or pallet jack
■
Strong enough to move a somewhat heavy object
■
Competent to level a heavy object
■
Meticulous inspection of thousands of very small objects
■
Perform electrical testing
■
Work in synchronization with others
■
Connect cables
■
Use computers
37
■
Perform software tasks of an administrative nature
Installer Education and Experience
The following list describes education and experience that is beneficial when
installing a Sun Datacenter Switch 3456:
■
Basic hand tool use
■
Basic mathematics
■
Basic physics
■
Basic electricity
■
Basic networking
■
Basic system administration
■
High school diploma
■
Degree in engineering (preferred)
Administration Personnel
This section describes the personnel who administers or manages the Sun Datacenter
Switch 3456. This is a remote role.
Administration Responsibilities
The person who administers the Sun Datacenter Switch 3456 must be fully capable
of the following tasks:
38
■
Use of computers and networking equipment
■
Troubleshoot problems and implement solutions
■
Perform administrative tasks
■
Authorized to enable, disable, and change hardware operating parameters
■
Responsible for mission-critical and highly available systems
■
Competent to make decisions which impact others and operations
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Administrator Education and Experience
The following list describes education and experience that is beneficial when
administering a Sun Datacenter Switch 3456:
■
Advanced computing topics
■
Advanced networking topics
■
UNIX shell and other command line interfaces
■
Advanced system administration
■
BS degree in computer science (preferred)
Maintenance and Service Personnel
This section describes the personnel who services the Sun Datacenter Switch 3456.
Servicing the switch is a hands-on role.
Maintenance and Service Responsibilities
The person who services the Sun Datacenter Switch 3456 must be fully capable of
the following tasks:
■
Use of basic hand tools
■
Strong enough to lift and align a heavy object
■
Meticulous inspection of thousands of very small objects
■
Perform electrical testing
■
Connect cables
■
Interpret indicators
■
Use computers
■
Perform system administration tasks
■
Recognize problems and deduce solutions
Maintenance and Service Education and
Experience
The following list describes education and experience that is beneficial when
maintaining or servicing a Sun Datacenter Switch 3456:
Chapter 7
Personnel Requirements
39
40
■
Basic hand tool use
■
Basic mathematics
■
Basic physics
■
Basic electricity
■
Basic networking
■
Basic system administration
■
College degree (engineering preferred)
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
CHAPTER
8
Site Planning Checklist
This section provides a checklist of items to be completed before installation of the
Sun Datacenter Switch 3456 switch. Topics include:
■
“Thermal Checklist” on page 41
■
“Power Checklist” on page 41
■
“Network Checklist” on page 42
■
“Location Checklist” on page 42
■
“Personnel Checklist” on page 43
Thermal Checklist
■
____ Are the environmental systems capable of supporting the Sun Datacenter
Switch 3456’s 40KW heat load requirement?
■
____ Has additional cooling capacity been installed, if necessary?
■
____ Have ducting and other delivery devices been installed to provide cooled air
to the Sun Datacenter Switch 3456?
■
____ Has the effect of the heat output from the Sun Datacenter Switch 3456 on
other systems been considered?
Power Checklist
■
____ Is there enough available power to support the Sun Datacenter Switch 3456?
■
____ Has the power been properly routed and conditioned to the final location?
41
■
____ Has the cabling, conduit, and supporting hardware been installed and is the
hardware up to local code?
Network Checklist
■
____ Has the network topology been determined?
■
____ Has the hardware to support the IB cables been specified and installed?
■
____ Has the length of each IB cable been determined?
■
____ Have the IB cables been properly routed and bundled?
Location Checklist
42
■
____ Is there a forklift or pallet jack available for moving the shipping crates?
■
____ Are the proper tools available for installation?
■
____ Is there enough space to unpack the Sun Datacenter Switch 3456 from the
shipping crate?
■
____ Have you considered the weight of the Sun Datacenter Switch 3456 and its
loading on the floor at the final location?
■
____ Was reinforcement of the floor at the final location done, if necessary?
■
____ Have you determined a path to the final location that can accommodate the
Sun Datacenter Switch 3456’s size and weight, along with the personnel moving
the switch?
■
____ Has the path to the final location been prepared for the Sun Datacenter
Switch 3456 transport?
■
____ Is there enough working space at the final location?
■
____ Have the administration servers been determined?
■
____ Have the administration servers been configured?
■
____ Has a pin repair kit been acquired?
■
____ Have the cable connection tools been acquired?
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Personnel Checklist
■
____ Have the installation personnel been determined?
■
____ Has the Sun Datacenter Switch 3456 administrator been determined?
■
____ Have the Sun Datacenter Switch 3456 service personnel been determined?
Chapter 8
Site Planning Checklist
43
44
Sun Datacenter Switch 3456 Site Planning Guide • August 2008
Index
A
administration
equipment, 22
interfaces, 23
personnel, 38
education and experience, 39
responsibilities, 38
air flow, 26
B
bus power, 5
C
cables
bundling, 32
connection, 36
delivery
floor and underfloor, 32
overhead, 32
length, 33
path length, 34
types, 33
cautions, 1
activity space, 3
cable, 31
electrical, 4
heat, 3
air, 3
aisles, 4
surface, 4
weight, 1
moving, 1
stationary, 2
chassis
dimensions, 9
weight, 7
checklist, 41
location, 42
network, 42
personnel, 43
power, 41
thermal, 41
components
dimensions, 9
distribution, 15
weight, 8
configuration
base, 13
complete, 13
component distribution, 15
considerations, 14
filler panels, 14
line cards, 14
power supplies, 14
incremental, 15
suggested, 16
D
dimensions
chassis, 9
components, 9
H
heat
density, 27
load, 25
45
I
incremental configuration, 15
installation
personnel, 37
education and experience, 38
responsibilities, 37
L
location
checklist, 42
equipment
for administration, 22
for movement, 17
installation
floor, 21
working space, 22
path of transport, 18
carpeting, 19
doors, 20
elevators, 20
grades, 19
security, 18
signage, 19
surface, 20
thresholds and gaps, 20
turning, 21
power, 5
repair kit, 23
power
backup, 28
bus, 5
checklist, 41
conduit, 29
consumption by component, 11
delivery, 28
main, 4
pin, 5
S
service personnel, 39
education and experience, 39
responsibilities, 39
specifications
dimensions, 8
chassis, 9
components, 9
environmental, 10
power, 11
weight, 7
chassis, 7
components, 8
suggested configuration, 16
M
T
main power, 4
maximum line cards per power supplies, 15
thermal
checklist, 41
map, 27
N
network checklist, 42
W
P
weight
chassis, 7
personnel
administration
education and experience, 39
responsibilities, 38
checklist, 43
installation
education and experience, 38
responsibilities, 37
service
education and experience, 39
responsibilities, 39
pin
46
components, 8
Sun Datacenter Switch 3456 Site Planning Guide • August 2008