Advertisement
Advertisement
CS400-AC Hardware Replacement Procedures
HR90-2009
Contents
Contents
HR90-2009 CS400-AC Hardware Replacement Procedures 2
About the CS400-AC Hardware Replacement Procedures
About the CS400-AC Hardware Replacement Procedures
The CS400-AC Hardware Replacement Procedures lists and shows GreenBlade, Subrack, processor/DIMM, and drive parts for the system. This manual includes procedures for replacing components in GB522 blades and expansion blades. It also includes procedures for replacing parts in the subrack. Cray personnel or customers who perform these replacement procedures must first complete Cray CCS hardware training and observe ESD precautions when servicing this equipment.
Document Versions
HR90-2009: April 2016.
ESD Precautions
Observe electrostatic discharge (ESD) precautions during the entire removal and installation process. Required apparel includes an ESD smock, ESD shoes, and an ESD wrist strap.
CAUTION:
● ESD Precautions
● Observe all ESD precautions. Failure to do so can result in equipment damage.
ESD Smock
Wear a Cray approved static-dissipative smock when you service or handle an ESD-sensitive device. Completely button the smock and wear it as the outermost layer of clothing. You must have a portion of the smock’s sleeves in direct contact with the skin of your arms. Skin contact is essential for a dissipative path-to-earth ground through your wrist strap. Tuck hair that exceeds shoulder length inside the back of the smock.
ESD Shoes
Wear approved static-dissipative shoes or approved dissipative heel straps on both shoes when you service or handle an ESD-sensitive device. When sensitive equipment is exposed to static discharge, ESD shoes provide a backup to the wrist straps and grounding cords and help prevent an excessive charge from building up on you when you contact the conductive flooring. Use dissipative footwear in addition to, not as an alternative to, a wrist strap.
ESD Wrist-strap
Wear a Cray approved wrist strap when you service or handle an ESD-sensitive device to eliminate possible ESD damage to equipment. Connect the wrist strap cord directly to earth ground.
Feedback
Visit the Cray Publications Portal at https://pubs.cray.com
and use the "Contact us" link in the upper-right corner to make comments online. Your comments and suggestions are important to us. We will respond to them within 24 hours.
HR90-2009 CS400-AC Hardware Replacement Procedures 3
CCS Parts Repair and Replacement
CCS Parts Repair and Replacement
Cray Cluster Supercomputer (CCS) Support is available to customers of Cray CS300, CS400, and CS-Storm systems. Four service tiers are available to CCS customers: Depot, Depot Rapid Exchange, Premium and
Premium Plus.
CCS customers should initiate a request for support by creating a case in the CrayPort portal. For Premium and
Premium Plus customers, support outside of local business hours can also be initiated by calling the after-hour support numbers.
This section provides information on the following service related topics:
● Requesting Part Repair or Replacement
● Spare Part Numbers
Requesting Part Repair or Replacement
Cray Cluster Supercomputer (CCS) customers and field support personnel must run appropriate diagnostics and perform troubleshooting before requesting part repair or replacement. These prerequisites are for replacement or repair of defective/suspect parts covered under a Cray Service Level Agreement (SLA). To streamline the return process, these prerequisites are being aligned with those of Cray’s vendors. Cray Service will assist customers with troubleshooting and collaborate with vendors as needed. Cray remains the sole authority in granting a Return
Materials Authorization (RMA). Cray may grant an RMA without diagnostic results for specific sites due to their security policies, or by authorization from a Cray executive, or for certain "Critical" cases as defined in the Cray
Cluster Supercomputer Support Operations Handbook .
Troubleshooting Procedures and Requirements
Cray Service provides detailed troubleshooting requirements and procedures to diagnose and identify faulty hardware. These procedures are unique to specific part categories. Diagnostic utilities may be used to generate log files or output that must be sent to Cray Service when requested before an RMA request is processed. As a minimum, customers must demonstrate that the fault follows the suspect part. Cray will not repair or replace any returned part that does not comply with these troubleshooting requirements and procedures.
Return Materials Authorization Number
After an RMA is approved, Cray Service issues a unique RMA number for that part. A single RMA number is generated for each batch of identical parts. Orders containing different parts are issued different RMA numbers.
For example, a replacement of 3 GPUs will share the same RMA number, but a replacement of 1 GPU and 1
HCA will be issued two RMA numbers. Cray will not repair or replace any returned part without an RMA number.
Parts received by Cray without an RMA number will be returned as is.
Packaging and Shipping RMA Parts
All parts returned to Cray must be individually tagged and identified with the RMA number. They must be properly packaged to prevent shipping and ESD damage. If possible, customers should save and reuse Cray shipping boxes (in good condition) when returning parts. If this is not possible, customers should provide enough
HR90-2009 CS400-AC Hardware Replacement Procedures 4
CCS Parts Repair and Replacement packaging (antistatic bags and ESD foam) so there is at least 2 inches of threedimensional clearance between the part and the shipping box. Cray will not be responsible for any damage as a result of poor customer packaging and may return the damaged part as is, without repair or replacement. Customers may consolidate multiple RMA requests in a single CrayPort case. Similarly, customers may consolidate multiple RMAs in a single box, provided it meets Cray packaging recommendations.
Customers must print the CrayPort case number visibly and legibly on the exterior of the box or on the shipping address label.
CS400-AC Part Numbers
Part Number Description
GreenBlade\Subrack
101129600 SR5110 subrack, 5U 10X node. Includes L/R angle brackets and cable management bar
000-01601A
101129800
101117701
101210400
SR8204 subrack, 8U 4X node. Includes L/R angle brackets and cable management bar
SR8116 subrack, 8U 16X node. Includes L/R angle brackets and cable management bar
Chassis, GB522XA, PCIe3 X16 Slot 1, KP
Chassis, GB522N expansion blade, up to 2X K20/X/K40
101210500
101195200
Chassis, GB522N expansion blade, up to 2X K80
Chassis, GB822XA, 6X 2.5" HDD bay
002-00372B Fan module assembly, air cooled systems, (for SR5110/SR8204/SR8116 subracks)
23101-0295-01 Fan module replacement dual fans: 12V, 3000 rpm with cable assembly
22302-0270-01 Fan module: fan interface board (FIB) for 2 fans
002-00374C iSCB module (SR5110/SR8204/SR8116 subracks)
22302-0313-01 iSCB module PCBA - Ext Board, GB2 IO, 10/100, RJ45, reset/LED
22309-0005-04 iSCB module PCBA - CPU, GB2 IMX253, 16 node mgr, hot-swap (plugs to midplane)
002-00316A
150-00143B
Cable management bar assembly, GB2 (SR5110/SR8204/SR8116 subracks)
Power supply, GreenBlade 1630W, 1U, 277V input, 133A/12V 6A/5Vsb (SR5110/SR8204/
SR8116)
101139200
101243300
Riser card, 1U PCIe-3 x16, S2600KP/TP Slot 1
Subrack support angle assembly, left side, 25.945 +-0.75" (SR5110/SR8204/SR8116)
101243400 Subrack support angle assembly, right side, 25.945 +-0.75" (SR5110/SR8204/SR8116)
GPUs and Coprocessors
101219400 Cable Assembly, Twin-Ax, Slot 2, PCIEX16 GB522N/M
51010-3218-00 PCIe Cable Bracket, GB522MN (Used with twin-ax cable assy)
56000-0030-01 Gasket EMI, GreenBlade Expansion
22302-0412-00 GPU Node IFB, 4U MIC CoProcessor GB522M, 2 GPU Hot-Swap, KP
HR90-2009 CS400-AC Hardware Replacement Procedures 5
CCS Parts Repair and Replacement
Part Number Description
22302-0412-10 GPU Node IFB, 4U K20/40 GB522N, 2 GPU Hot-Swap, KP
22302-0412-20 GPU Node IFB, 4U K80 GB522N, 2 GPU Hot-Swap, KP
22302-0402-02 GPU Bridge Board/Riser, GB522 Expansion Blade, (MIC/PHI, K20/K40, K80)
22303-0082-01 GPU Front PCBA/LED/push button switch assembly (MIC/PHI, K20/K40, K80 expansion blades)
101210300 Chassis, GB522M Expansion Blade, with cover/cables/brackets, up to 2X MIC/PHI (not incl.)
101210400
101210500
100972402
101142501
Chassis, GB522N Expansion Blade, with cover/cables/brackets, up to 2X K20/K40 (not incl.)
Chassis, GB522N Expansion Blade, with cover/cables/brackets, up to 2X K80 (not incl.)
GPU Card, TESLA ATLAS K40 12GB 235W Passive
GPU Card, STELLA DUO K80 24GB 300W Passive
130-00149A MIC Card, XEON PHI 7120P Passive 61C 1.25GHZ 16GB 300W
Rackmount Servers (2U)
101137200
101125300
Chassis, 2828X, 2U S2600WT2 dual Xeon 8X 2.5" HDD, up to 2X 1100W (Intel: 936032,
R2208WT2YS)
Chassis, 2828X, 2U S2600WTT dual Xeon 8X 3.5" HDD, up to 2X 1100W (Intel: 936035,
R2308WTTYS)
101142900
150-00145A
190-00137A
100993600
Power supply, 1100W, 80+ platinum (2828X)
Power supply, 1600W, 80+ platinum (2820XT)
Rail kit, 1U/2U, full extension & tool-less, 609.6-762MM 800MM MAX (2828X)
RMM4 lite remote management module, KVM
Motherboards, DIMMs and Processors
101123200 Intel Kennedy Pass motherboard, dual Xeon Haswell, 8X DIMM, FDR IB (Connect-IB) onboard
101132800
101123800
101132900
101181500
101181600
167-00509A
100944700
100961800
100961801
100961802
101107700
Intel Kennedy Pass motherboard, dual Xeon Haswell, 8X DIMM, (No InfiniBand on-board)
Intel Taylor Pass motherboard, dual Xeon Haswell, 16X DIMM, QSFP FDR IB
Intel Taylor Pass motherboard, dual Xeon Haswell, 16X DIMM, (No InfiniBand on-board)
Heatsink, 1U Passive, Std - 91.5 x 91.5, 47 fins, AL base, heatpiped/copper fins
Heatsink, 1U Passive, Wide - 91.5 x 110, 57 fins AL base, heatpiped/copper fins
Battery backup, RAID smart lithium-ion 1500 mAh
DIMM, 8GB DDR4-2133 Reg dual rank 2RX8
DIMM, 16GB DDR4-2133 Reg dual rank ECC 1.2V 1GX4
DIMM FRU,16GB DDR4-2133 REG DUAL RANK ECC 1.2V 1GX4 (HYNIX)
DIMM FRU,16GB DDR4-2133 REG DUAL RANK ECC 1.2V 1GX4 (SAMSUNG)
DIMM, 32GB DDR4-2133 Reg dual rank ECC 1.2V 1GX4
HR90-2009 CS400-AC Hardware Replacement Procedures 6
CCS Parts Repair and Replacement
Part Number
101159100
101169900
101187200
101245301
101278300
101078600
101078700
101078900
101079000
101079100
101120400
Disk Drives
101030500
101032700
101178400
101178500
101178600
101178700
101125800
101178800
101179000
101179900
101180400
101180700
101180800
101180900
101181000
101181100
101181200
101205700
101206400
101206600
101206700
Description
IC, processor, E5-2660 V3, 2.6 GHz 10C Haswell 105W
IC, processor, E5-2620 V3, 2.4 GHz 6C Haswell 85W
IC, processor, E5-2683 V3, 2.40 GHz 14C Haswell 120W
IC, processor, E5-2699 V3, 2.3 GHz 18C Haswell 145W, SR1XD
IC, processor, E5-2667 V3, 3.2 GHz 8C Haswell 135W
IC, processor, E5-2670 V3, 2.3 GHz 12C Haswell 120W, M1
IC, processor, E5-2680 V3, 2.5 GHz 12C Haswell 120W, M1
IC, processor, E5-2690 V3, 2.6 GHz 12C Haswell 135W, M1
IC, processor, E5-2695 V3, 2.3 GHz 14C Haswell 120W, C1
IC, processor, E5-2650 V3, 2.3 GHz 10C Haswell 105W, M1
IC, processor, E5-2698 V3, 2.3 GHz 10C Haswell 135W, 40M , C1
Disk drive, SATA 600 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 480 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 120 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 160 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 240 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 300 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SAS 300GB 10K 6 2.5" 6Gb/s 64MB, Savvio
Disk drive, SATA 800 GB SSD, 2.5" 6 Gb/s, 20nm, Intel S3500
Disk drive, SATA 100 GB SSD, 2.5" 6 Gb/s, 25nm, Intel S3700
Disk drive, SATA 200 GB SSD, 2.5" 6 Gb/s, 25nm, Intel S3700
Disk drive, SATA 800 GB SSD, 2.5" 6 Gb/s, 25nm, Intel S3700
Disk drive, MSATA 120 GB SSD, 2.5" 6 Gb/s, 25nm, Intel 525
Disk drive, MSATA 180 GB SSD, 2.5" 6 Gb/s, 25nm, Intel 525
Disk drive, MSATA 240 GB SSD, 2.5" 6 Gb/s, 25nm, Intel 525
Disk drive, MSATA 60 GB SSD, 2.5" 6 Gb/s, 25nm, Intel 525
Disk drive, 400 GB SSD, 1/2 Height PCIe 2.0, 25nm, Intel 910
Disk drive, 800 GB SSD, 1/2 Height PCIe 2.0, 25nm, Intel 910
Disk drive, SATA 180 GB SSD, 2.5" 6 Gb/s, 20nm, R540/W490, Intel Pro 2500
Disk drive, SATA 120 GB SSD, 2.5" 6 Gb/s, 20nm, R540/W490, Intel Pro 2500
Disk drive, SATA 240 GB SSD, 2.5" 6 Gb/s, 20nm, R540/W490, Intel Pro 2500
Disk drive, SATA 480 GB SSD, 2.5" 6 Gb/s, 20nm, R540/W490, Intel Pro 2500
HR90-2009 CS400-AC Hardware Replacement Procedures 7
CCS Parts Repair and Replacement
Part Number
101238200
160-00184B
160-00185C
160-00228A
160-00258A
PCIe Cards
100629400
100882500
101082500
101196800
101203000
101264800
101267000
101267100
101267200
101267300
101268500
101278400
101290000
101290100
131-00153A
132-00110B
132-00125A
132-00128A
132-00137B
132-00138B
132-00145A
132-00150A
132-00157A
132-00159A
Power Cords
085-00370B
Description
Disk drive, SATA 2 TB SSD, 2.5" 6 Gb/s, 7200 RPM, 128 MB Cache, Seagate Enterprise
Disk drive, SATA 500 GB SSD, 2.5" 6 Gb/s, 7200 RPM, 32 MB Cache, Seagate Constellation
Disk drive, SATA 250 GB SSD, 2.5" 6 Gb/s, 7200 RPM, 64 MB Cache, Seagate Constellation
Disk drive, SATA 1 TB SSD, 2.5" 6 Gb/s, 7200 RPM, 64 MB Cache, Seagate Constellation
Disk drive, SATA3 400 GB SSD, 2.5" 6 Gb/s, Read-500Mb/s, Write-460Mb/s, Intel
PCIe, Gen2 dual port 1GBE [GB822x] RJ45, 10/100/1GE, Intel 82576, (2.5 Gbps) includes bracket
PCIe3.0, x8, 40 GbE, HBA, dual-port, ConnectX-3, QSFP (MCX314A-BCBT)
PCIe3.0, x8, HCA, Connect-IB, single-port QSFP FDR, 56 Gb/s (MCB191A-FCAT)
PCIe3.0, x8, 40 GbE, HBA, single-port QSFP (MCX313A-BCBT)
PCIe3.0, 800 GB SSD, 1/2 height, 20 nm, R2800/W1900, P3700
PCIe3.0, 1.6 TB SSD, 1/2 height, 20 nm MLC, HHHL, P3700
PCIe3.0, 2.0 TB SSD, 1/2 height, 20 nm MLC, HHHL, P3700
PCIe3.0, 400 GB SSD, 1/2 height, 20 nm MLC, HHHL, P3700
PCIe3.0, 2.0 TB SSD, 1/2 height, 20 nm MLC, HHHL, P3600
PCIe3.0, 1.6 TB SSD, 1/2 height, 20 nm MLC, HHHL, P3600
PCIe3.0 x16, HCA ConnectX-4 VPI single-port QSFP, EDR 100 Gb/s InfiniBand
PCIe3.0 x16, HCA ConnectX-4 VPI dual-port QSFP, EDR 100 Gb/s InfiniBand
PCIe3.0 x16, HCA ConnectX-4 single-port QSFP, EDR 100 Gb/s InfiniBand
PCIe3.0 x16, HCA ConnectX-4 dual-port QSFP, EDR 100 Gb/s InfiniBand
PCIe, x8, RAID SAS 8P Ctrl 0/1/5/6/10/50/60
PCIe2.0 x8, Intel True Scale Fabric InfiniBand, single-port, QSFP, QLE7300
PCIe2.0 x8, Intel True Scale Fabric InfiniBand, dual-port, QSFP, QLE7300
PCIe2.0 x8, Ethernet X520-DA2, dual-port, 10GB SFP+, low profile, full height
PCIe3.0 x8, HCA ConnectX-3 VPI single-port QSFP, 40Gb/s QDR IB, 10GbE
PCIe3.0 x8, HCA ConnectX-3 VPI dual-port QSFP, 40Gb/s QDR IB, 10GbE
PCIe3.0 x8, HCA ConnectX-3 VPI single-port QSFP, 40Gb/s FDR IB, 40GbE
PCIe3.0 x8, HCA ConnectX-3 EN dual-port SFP+, 10Gb/s, 40GbE
PCIe3.0 x4, HCA ConnectX-3 EN single-port SFP+, 10Gb/s, 10GbE
PCIe3.0 x16, HCA Connect-IB dual-port 56Gb/s FDR
Power cord, LS14 to LS26, 10A-250/277V, 1.2 m (4 ft)
HR90-2009 CS400-AC Hardware Replacement Procedures 8
CCS Parts Repair and Replacement
Part Number
085-00374A
085-00382A
085-00383A
085-00218A
085-00251A
085-00361A
085-00360A
085-00294A
085-00384A
085-00273A
085-00351A
Description
Power cord, LS14 to LS26, 10A-250/277V, 1.5 m (5 ft)
Power cord, LS25 to LS26, 250/277V, 1.2 m (4 ft)
Power cord, LS25 to LS26, 250/277V, 1.5 m (5 ft)
Power cord, C14 to C13, 17AWG, 10A-250V, 1.5 m (5 ft)
Power cord, C14 to C13, 17AWG, 10A-250V, 1.2 m (4 ft)
Power cord, C14 to C13, 17AWG, 10A-250V, 1.8 m (6 ft)
Power cord, C14 to C13, 17AWG, 10A-250V, 3.0 m (10 ft)
Power cord, C14 to C19, 14AWG, 15A-250V, 1.8 m (6 ft)
Power cord, C14 to C19, 14AWG, 13A-250V, 3.0 m (10 ft)
Power cord, C20 to C13, 10A-250V, 1.8 m (6 ft)
Power cord, C20 to C13, 15A-250V, 1.5 m (5 ft)
HR90-2009 CS400-AC Hardware Replacement Procedures 9
Power Cords
Power Cords
The following power cords are used with CS400-AC systems.
1.2 m (47 in.)
1.5 m (59 in.)
LS-14
LS-25
C14
LS14 to LS26, 250-277 V, 10 A
1.2 m (47 in.)
1.5 m (59 in.)
LS25 to LS26, 250-277 V
1.2 m (47 in.)
1.5 m (59 in.)
1.8 m (6 ft)
3.0 m (10 ft)
C14 to C13, 250 V, 10 A, 17 AWG
1.8 m (6 ft) - 15 A
3.0 m (10 ft) - 13 A
C14
C20
C14 to C19, 250 V, 14 AWG
1.8 m (6 ft) - 10 A
1.5 m (5 ft) - 15 A
C20 to C13, 250 V
HR90-2009 CS400-AC Hardware Replacement Procedures
LS-26
LS-26
C13
C19
C13
10
PCI Card Brackets
PCI Card Brackets
A custom PCI mounting bracket is required to install most PCI expansion cards in a GreenBlade chassis. The following custom Intel and Mellanox PCI brackets are available for popular expansion/add-on cards. If a bracket is not listed for the card that you want to add, please contact your Cray sales representative.
HR90-2009 CS400-AC Hardware Replacement Procedures 11
PCI Card Brackets
Brkt: 004-01173C (51010-2346-02)
Brkt & Gasket Kit: 002-00405A
Mellanox ConnectX-2: MNPH29C/D-XTR,
MCX312B-XCCT, MCX312A-XCBT
Brkt: 004-01172C (51010-2396-02)
Brkt & Gasket Kit: 002-00399A
Mellanox ConnectX-3 VPI: MCX353A-QCBT/TCBT/FCBT
Brkt: 101304400
Brkt & Gasket Kit: 101304500
Mellanox ConnectX-4: MCX456A-ECAT
Brkt: 101304600
Mellanox ConnectX-4 VPI EDR: MCX455A-ECAT
Brkt: 004-01222C (51010-2395-02)
Brkt & Gasket Kit: 002-00397A (90301-0418-00)
Mellanox Connect-IB, ConnectX-2/3: MCB192A-FCAT,
MCB194A-FCAT, MCX354A-QCBT/TCBT/FCBT,
MHQH29C-XTR, MHRH29C-XTR
Brkt: 51010-3056-10
Brkt & Gasket Kit: 101224600
Mellanox Connect-IB FDR: MCB191A-FCAT,
MCB193A-FCAT
Brkt: 004-01318A (51010-2396-10)
Brkt & Gasket Kit: 002-00398A (90301-0411-00)
Mellanox ConnectX-3: MCX353A-QCBT
Brkt: 004-01317A (51010-2749-00)
Brkt and Gasket Kit: 90301-0426-00
Mellanox ConnectX-3 10GbE: MCX311A-XCCT/XCAT
HR90-2009 CS400-AC Hardware Replacement Procedures 12
GB522 Overview and Replacement Videos
GB522 Overview and Replacement Videos
The following list provides links to a series of GB522 videos. Most of these instructional videos are less than a minute in length. For those videos that are longer, the length is indicated next to the title. These videos are provided through unlisted links on YouTube TM , so they will not show up in searches or other lists.
GB522 Overview Video
● GB522 Blade Overview (5:52)
GB522 Compute Blade Videos
● Remove and Install Blade
● Remove and Install Cover
● Replace Disk Drive
● Replace Motherboard (4:55)
● Remove CPU
● Install CPU
● Remove Bridge Board
● Install Bridge Board
GB522 Expansion Blade Videos
● Remove from Subrack
● Install in Subrack
● Remove Accelerator
● Install Accelerator
● Separate Expansion Blade and Compute Blade
● Join Expansion Blade and Compute Blade
HR90-2009 CS400-AC Hardware Replacement Procedures 13
GB522X Replacement Procedures
GB522X Replacement Procedures
The following procedures are included in this section to prepare a GB522X chassis before replacing failing components/assemblies. The replacement procedures in this CS400-AC Hardware Replacement Procedures document begin with the GB522X chassis removed from the subrack and placed on an ESD safe work surface with the cover removed. These preparation procedures are not repeated in each component/assembly replacement procedure.
● GB522X Power Down
● GB522X Chassis Removal and Installation
● GB522X Cover Removal
● GB522X Cover Installation
● GB522X Exploded Views
● PCIe Card Installation
● HDD/SSD Replacement
NOTE: Short instructional videos for many of the GB522X replacement procedures are available. Refer to this page for a complete list of all the videos:
GB522 Overview and Replacement Videos on page 13.
GB522X Power Down
1.
Power down the GB522X using ACE, IPMI, or iSCB commands.
2.
Check that the power status LED on the front is off.
3.
Disconnect all Ethernet, InfiniBand (IB), and other I/O cables from the rear of the blade.
GB522X Chassis Removal and Installation
First, power down the GB522X and disconnect all cables.
Removal
1.
Lift up the ejector handle on the bottom of the GB522X. Refer to the following figure.
2.
Pull the ejector handle to unseat the GB522X from the subrack midplane.
3.
Remove the GB522X from subrack and place it on an ESD-safe work surface.
HR90-2009 CS400-AC Hardware Replacement Procedures 14
GB522X Replacement Procedures
COM
GreenBlade
Installation
1.
Slide the GB522X into the subrack, and carefully push it in until it seats with the midplane connectors.
2.
Power on the GB522X and verify that the blue LED lights to indicate power is applied to the blade.
3.
Connect the cables to the GB522X, boot the operating system, and close the rear doors of the cabinet.
GB522X Cover Removal
1.
Remove the screw that secures the cover to the chassis. Refer to the following figure.
2.
Press down on the latch button and slide the cover toward the rear of the chassis.
3.
Lift and remove the cover.
HR90-2009 CS400-AC Hardware Replacement Procedures 15
GB522X Replacement Procedures
GB522X Cover Installation
The top cover is a required component of the GB522X GreenBlade chassis. Do not attempt to insert a
GreenBlade chassis into a subrack without the top cover in place.
1.
Tip the front of the cover down so the flange on the cover fits below the top of the front panel housing.
2.
Lower the back of the cover. The cutouts on the side fit around the pins/rivets inside the chassis and the pins on the underside of the cover fit into the recesses in the rear panel.
3.
Slide the cover forward and secure it to the chassis with one screw.
TO
W
ARD
TO
W
ARD
GB522X Exploded Views
The following three exploded views of GB522X models show the internal components of each chassis. Refer to these illustrations when replacing components and following procedures. These detailed exploded views show screw locations, component placement, and the intended assembly of components. Detailed step-by-step information describing the location of components and screws for component removal and replacement is not provided in all the replacement procedures in this CS400-AC Hardware Replacement Procedures document.
Views of the GB522M/N expansion blades are provided in GB522M and GB522N Exploded Views on page 28.
HR90-2009 CS400-AC Hardware Replacement Procedures 16
GB522X Replacement Procedures
HR90-2009 CS400-AC Hardware Replacement Procedures 17
(CS-GB-522X [Second generation])
GB522X Replacement Procedures
SATA-3 cable assembly
(between bridge PCB and bracket)
Bridge board assembly
(same as 1st generation)
Disk high-mount bracket
HDD/SSD
SATA power cable (5-4 pin)
SATA-3 cable assembly
(from SATA 0 on motherboard)
Bridge board assembly
(same as 1st generation)
PCIe Card Installation
Prerequisites
● Refer to
CS400-AC Part Numbers on page 5 for available PCIe add-on cards.
● Refer to
PCI Card Brackets for a list of mounting brackets to be used with the available PCIe add-on cards.
About this task
This procedure describes how to install an add-on card into a GreenBlade chassis. Refer to the corresponding step numbers in the following figures. To remove a PCIe card, follow the steps in reverse.
Procedure
1.
If necessary, remove the blank PCI slot bracket from the front panel.
HR90-2009 CS400-AC Hardware Replacement Procedures 18
GB522X Replacement Procedures
2.
Many PCIe cards require removal of the manufacturer's bracket before installing the card in a GreenBlade chassis. Refer to the figure below.
3.
Attach a mounting bracket and EMI gasket to the PCI card.
A custom Cray PCI mounting bracket is needed to install most PCI cards into a GreenBlade chassis. Cray has custom brackets and EMI gaskets for supported add-on cards.
EMI gasket
Remove manufacturer’s bracket
4.
Attach the slot 1 PCI riser card to the add-on card as shown.
5.
Install the PCI card assembly into the chassis by inserting the riser card into the open PCI slot 1 connector.
HR90-2009 CS400-AC Hardware Replacement Procedures 19
GB522X Replacement Procedures
6.
Secure the bracket to the front panel using two M3-0.50 x 4 mm screws. Check that the EMI gasket and bracket are not obstructing any LEDs or light pipes.
HDD/SSD Replacement
The following procedures describe how to replace an internal HDD/SSD in the two GB522X chassis models.
● First generation
● Second generation (with high-mount drive bracket)
For many of the steps, refer to the corresponding number in the figure for each procedure.
GB522X SSD Replacement (High-Mount Bracket)
Removal
1.
Remove the GreenBlade chassis from the subrack and place it on a stable ESD-safe work surface. Remove the cover.
2.
Disconnect the SATA power connector (5-4 pin) from the drive.
3.
Disconnect the SATA signal (flexible-foil) connector from the drive.
4.
Remove the four screws that attach the high-mount bracket to the base of the chassis. Two screws go through the node IFB.
HR90-2009 CS400-AC Hardware Replacement Procedures 20
GB522X Replacement Procedures
There are three 6-32 x 0.5 inch screws and one 6-32 x 0.25 inch screw.
5.
Carefully remove the mounting bracket and HDD/SSD from the chassis.
6.
Remove the four screws that attach the HDD/SSD to the mounting bracket.
7.
Remove the HDD/SSD from the mounting bracket.
Installation
8.
Install the replacement HDD/SSD and attach it to the bracket with the four M3 x 4.0 screws.
9.
Attach the high-mount bracket/drive assembly to the chassis.
10.
Connect the SATA signal and power connectors.
GB522X HDD Replacement (First Generation Chassis)
Removal
1.
Remove the GreenBlade chassis from the subrack and place it on a stable ESD-safe work surface. Remove the cover.
2.
Remove the screw that attaches the drive bracket to the base of the chassis.
HR90-2009 CS400-AC Hardware Replacement Procedures 21
GB522X Replacement Procedures
3.
Carefully unseat the bracket and drive assembly from the connector on the node IFB and remove the assembly from the chassis.
4.
Remove the four screws that attach the HDD to the drive bracket. (Screws could be through the top or side of the bracket depending on the HDD model.)
5.
Remove the HDD from the drive bracket.
Installation
6.
Install the replacement HDD and attach it to the drive bracket with the four M3 x 4.0 screws.
7.
Position the HDD bracket assembly in the base of the chassis. Then seat the drive to the connector on the node IFB.
8.
Attach the HDD bracket assembly to the chassis (one 6-32 x 0.25 screw).
HR90-2009 CS400-AC Hardware Replacement Procedures 22
GB522M/N Replacement Procedures
GB522M/N Replacement Procedures
The following procedures are included in this section to prepare a GB522M/N expansion blade before replacing failing components/assemblies. The replacement procedures in this CS400-AC Hardware Replacement
Procedures document begin with the GB522MN chassis removed from the subrack and placed on an ESD safe work surface with the cover removed. These preparation procedures are not repeated in each component/ assembly replacement procedure.
● GB522M/N Power Down
● GB522M/N Chassis Removal
● GB522M/N Cover Removal
● GB522M/N Cover Installation
● GB6522M/N: Separate/Join Expansion and Compute Blades
● GB522M and GB522N Exploded Views
● GPU/Coprocessor Replacement
NOTE: Short instructional videos for many of the GB522M/N replacement procedures are available. Refer to this page for a complete list of all the videos: GB522 Overview and Replacement Videos on page 13.
GB522MN Expansion Blade Removal
1.
Power down the expansion blade using ACE, IPMI, or iSCB commands.
2.
Check that the power status LEDs on both the compute blade and attached expansion blade are off.
3.
Disconnect all Ethernet, InfiniBand (IB), and other I/O cables from the rear of the compute blade.
4.
Lift up the ejector handle on the bottom of the GB522 compute blade. Refer to the following figure.
5.
Pull the ejector handle to unseat the GB522 from the subrack midplane and pull/remove the GB522 from the subrack.
HR90-2009 CS400-AC Hardware Replacement Procedures 23
6.
Set the expansion blade on an ESD-safe work surface.
GB522M/N Replacement Procedures
COM
GreenBlade
GB522MN Cover Removal
1.
Remove the two screws that secure the cover to the expansion chassis. Refer to the following figure.
2.
Press down on the latch button and slide the cover toward the rear of the chassis.
3.
Lift and remove the cover.
TOW
ARD
HR90-2009 CS400-AC Hardware Replacement Procedures 24
GB522M/N Replacement Procedures
GB522MN Cover Installation
As with all GreenBlade chassis, the top cover of the GB522M and GB522N expansion blades is a required component. Do not attempt to insert a GreenBlade chassis into a subrack without the top cover in place.
1.
Place the cover on the expansion blade. The cutouts on the side fit around the pins/rivets inside the chassis.
The pins on the underside of the cover fit into the recesses in the rear panel.
2.
Slide the cover forward until it latches.
3.
Secure the cover to the chassis with two screws.
THE
REAR
GB522MN: Separate/Join Expansion and Compute Blades
Separate the GB522MN Expansion Blade
HR90-2009 CS400-AC Hardware Replacement Procedures 25
GB522M/N Replacement Procedures
1.
Remove the 6 screws that secure the expansion blade to the compute blade as shown in the following figure.
THE
REAR
2.
Unseat the connectors at the front (handle end) of the blade first.
3.
Then unseat the connectors at the rear of the blade.
4.
Pull the blades apart by lifting the expansion chassis straight up.
Remove 6 screws.
HR90-2009 CS400-AC Hardware Replacement Procedures 26
Unseat front connectors
(Bridge board PCIe).
TO
W
W
ARD
ARD
GB522M/N Replacement Procedures
Lift straight up.
Pull blades apart.
Join the Blades
1.
Position the expansion blade over the compute blade, and line up the PCI connectors and chassis guides.
2.
Mate each of the connectors and the two chassis. Press down firmly on the expansion blade.
3.
Install 6 screws to secure the expansion blade to the compute blade.
HR90-2009 CS400-AC Hardware Replacement Procedures 27
GB522M/N Replacement Procedures
GB522M and GB522N Exploded Views
The following three exploded views of GB522 expansion blades show the internal components of each GPU/ coprocessor chassis. Refer to these illustrations when replacing components and following procedures. These detailed exploded views show screw locations, component placement, and the intended assembly of components.
Detailed step-by-step information describing the location of components and screws for component removal and replacement is not provided in all the replacement procedures in this CS400-AC Hardware Replacement
Procedures document.
HR90-2009 CS400-AC Hardware Replacement Procedures 28
(CS-GB-522M-P245W)
GB522M/N Replacement Procedures
Air duct GPU bracket
MIC/coprocessor card
MIC GPU mounting bracket - rear
MIC GPU mounting bracket - front
Twin-ax cable assembly
(slot 2, PCIe X16)
GB522M MIC chassis assembly
GPU gasket
GPU front LED/switch cable assembly
GPU MIC power cable assembly
(10P to 8P & 6P)
GPU bridge board/riser
Air duct riser bracket
GB522 GPU bridge board (KP)
Power connectors to rear GPU
(8P and 6P)
HR90-2009 CS400-AC Hardware Replacement Procedures 29
GB522M/N Replacement Procedures
(CS-GB-522N)
Air duct GPU bracket
K20/K40 card
K20/K40 mounting bracket - rear
K20/K40 mounting bracket - front
Twin-ax cable assembly
(slot 2, PCIe X16)
GB522N chassis assembly
GPU gasket
GPU front LED/switch cable assembly
K40 power cable assembly
(10P bridgeboard to 6P K40)
GPU bridge board/riser
Air duct riser bracket
GB522MN GPU node board
Power connector, rear GPU (6P)
GPU/Coprocessor Replacement
Refer to the exploded views of the expansion blades when following these procedures:
on page 28.
Remove
1.
Disconnect the power cable connector(s) from the GPU/Coprocessor.
2.
For the rear GPU/Coprocessor (near the GPU node board), remove the air duct GPU bracket.
3.
Remove the 2 screws securing the GPU/Coprocessor brackets to the chassis.
HR90-2009 CS400-AC Hardware Replacement Procedures 30
GB522M/N Replacement Procedures
4.
Carefully unseat the GPU/Coprocessor from the PCIe connector. Remove the GPU/Coprocessor.
Install
1. New GPU/Coprocessor cards . New cards require removal of the manufacturer's PCI bracket. Some cards may also have a front support bracket that needs to be removed.
2.
Install the front and rear GreenBlade mounting brackets to the replacement GPU/Coprocessor.
3.
Connect the power cable connector(s) to the GPU/Coprocessor (do this before placing the card in the chassis).
4.
Position the GPU/Coprocessor in the chassis and seat it to the PCIe connector.
5.
Secure the GPU/Coprocessor to the expansion blade chassis with 2 screws.
6.
For the rear GPU/Coprocessor, install the air duct GPU bracket.
HR90-2009 CS400-AC Hardware Replacement Procedures 31
Motherboard, Processor and DIMM Replacement
Motherboard, Processor and DIMM Replacement
The following procedures are included in this section:
● GB522X Motherboard Removal
● GB522X Motherboard Installation
● Processor and Heatsink Replacement
● DIMM Replacement
NOTE: Instructional videos showing motherboard and CPU replacement are available. Refer to this page for a complete list of all the GB522 videos:
GB522 Overview and Replacement Videos on page 13.
GB522X Motherboard Removal
Prerequisites
● The GreenBlade chassis is removed from the subrack and placed on a stable ESD-safe work surface. The cover is removed.
● Part Numbers:
○ Refer to
CS400-AC Part Numbers on page 5 for a list of all spare motherboards
About this task
This procedure describes how to replace a motherboard in a CS400-AC series GreenBlade chassis. Refer to
on page 16.
● Motherboards are shipped without DIMMs, processors, or heatsinks installed.
● We recommend swapping DIMMs, processors, and heatsinks from the defective motherboard to the replacement motherboard when both motherboards are on a stable ESD-safe work surface. This is the preferred method versus removing or replacing processors and DIMMs from a motherboard mounted in a
GreenBlade chassis.
● After the blade is returned to the system and booted, make sure the replacement motherboard uses the same firmware revision as the other motherboards in the system.
Remove PCIe Card and Bridge Board and Disconnect Cables
1.
If a PCIe card is used, remove the PCIe card assembly (card and riser).
Refer to
on page 18
2.
GB522X Second Generation (with SATA-3 cable between bridge PCBA and bracket).
HR90-2009 CS400-AC Hardware Replacement Procedures 32
Motherboard, Processor and DIMM Replacement
Disconnect the SATA-3 cable from the motherboard and from the HDD/SSD.
3.
Remove the bridge board and hex support screw.
4.
Disconnect the 2x7 fan control cable from the motherboard.
5.
Disconnect the two main power connectors from the motherboard.
Remove Motherboard
6.
Remove the 7 screws (6-32 x 5/16) that secure the motherboard to the chassis. (The 8th screw was the hex support screw for the bridge board that was already removed.)
7.
Lift the motherboard out of the chassis and place it on an ESD-safe work surface.
GB522X Motherboard Installation
About this task
GB522X Exploded Views on page 16.
Install Motherboard
1.
Add DIMMs and processors to the replacement motherboard.
2.
If necessary, remove the blank InfiniBand (IB) cover from the front panel.
3.
Check that an EMI gasket is in position on the inside of the front panel.
4.
Check that a motherboard insulator is in position in the bottom of the chassis and the standoffs are not obstructed.
5.
Install the motherboard into the chassis.
Tip/angle the front of the board down into the chassis so the I/O connectors fit into the openings in the front panel. Some pressure may be needed to compress the EMI gasket and align the holes in the motherboard with the standoffs in the chassis. Check that all I/O connectors are clear of any obstruction.
6.
Reinstall the bridge hex support screw.
7.
Secure the motherboard with seven 6-32 x 5/16 screws.
8.
Install the bridgeboard. For GB522X second generation blades, connect the SATA-3 cable to the motherboard and HDD/SSD.
9.
Connect the fan control and power connectors.
10.
Reinstall the PCIe card assembly.
HR90-2009 CS400-AC Hardware Replacement Procedures 33
Motherboard, Processor and DIMM Replacement
Processor and Heatsink Replacement
Prerequisites
● The 2600 motherboard is removed from the chassis and is placed on a stable ESD-safe work surface.
● Part Numbers:
○ Refer to
CS400-AC Part Numbers on page 5 for a list of all processor models, common supplies, and
tools.
● Tools:
○ Phillips screwdriver, #2
○ Vacuum pen
○ Plastic putty knife
● Supplies:
○ Isopropyl alcohol, 90% or higher concentration
○ Clean, lint-free cloth
○ Thermal paste/grease
About this task
This procedure describes how to replace a passive heatsink and processor from an Intel ® S2600 motherboard.
Motherboards are not shipped with processors installed. Processors, heatsinks, and memory DIMMs must be removed from the defective motherboard and installed on the replacement motherboard.
Remove Heatsink
The heatsink is attached to the processor socket with captive fasteners. Use a #2 Phillips screwdriver to loosen four screws located on the heatsink corners in a diagonal pattern.
1.
Start with one screw (A) and loosen it by giving it two rotations. Refer to the following figure.
2.
Proceed to loosen the remaining screws (B, C, and D) by giving each two rotations. Repeat this process by loosening each screw two rotations, each time, until all screws are loosened.
3.
Lift the heatsink straight up.
HR90-2009 CS400-AC Hardware Replacement Procedures 34
Motherboard, Processor and DIMM Replacement
Clean Heatsink and Processor
4.
Use a plastic putty knife to remove excess thermal interface material (paste/grease) from the heatsink.
5.
Clean the heatsink with a clean, lint-free cloth dampened with isopropyl alcohol. Make sure the cloth is not dripping. Do not use paper towel as it will leave lint behind.
Plastic putty knife
Clean, lint-free cloth dampened with isopropyl alcohol
6.
Clean the processor with a new cloth dampened with isopropyl alcohol. Take your time to prevent any alcohol or thermal paste debris from contaminating the processor socket.
7.
Use a dry cloth to dry the processor and heatsink so they are ready for a new application of thermal paste.
Remove Processor
8.
Unlatch the processor load plate as shown in the following figure.
a. First, release the lever handle marked with the "Unlock (1)" symbol b. Next, release the second lever handle.
HR90-2009 CS400-AC Hardware Replacement Procedures 35
Motherboard, Processor and DIMM Replacement
9.
Open the load plate.
a. Lift the load plate.
b. For a replacement motherboard, open the latch taking care not to touch any of the pins inside the socket.
c. Remove the socket cover from the load plate by pressing it out.
Replacement motherboards do not include processors. There should be a socket cover (C).
Save and reuse the socket cover if the processor needs to be removed in the future.
Socket with processor.
10.
Remove the processor. Cray Service recommends using a vacuum pen to install and remove processors.
a. Carefully lift the processor out of the socket. DO NOT drop the processor on the socket pins.
b. Immediately place your other hand underneath the processor to protect the socket pins. Place the processor on an ESD-safe work surface.
Immediately place hand under processor to protect socket pins
HR90-2009 CS400-AC Hardware Replacement Procedures 36
Motherboard, Processor and DIMM Replacement
Install Replacement Processor
11.
Install the replacement processor.
a. If necessary, remove the processor from its packaging. Carefully remove the protective cover from the bottom side of the processor, taking care not to touch any contacts.
b. Orient the processor with the socket so that the processor cutouts match the four orientation posts on the socket.
c. Note the location of a gold key at the corner of the processor.
d. Carefully place (DO NOT drop) the processor into the socket. Hold the processor down with your finger as you release the vacuum pencil.
12.
Close the load plate. Carefully lower the load plate down over the processor.
13.
Lock the load plate.
a. Push down on the locking lever marked with the "Lock (1)" symbol.
b. Slide the tip of the lever under the notch in the load plate. Make sure the load plate tab engages under the socket lever when fully closed.
c. Repeat the steps to latch the locking lever on the other side. Latch the levers in the order shown.
HR90-2009 CS400-AC Hardware Replacement Procedures 37
Motherboard, Processor and DIMM Replacement
Install Heatsink
14.
Apply thermal grease to the heatsink as shown. Apply 5 small dabs/drops of thermal grease in an X pattern.
Do not spread the grease. (If using a new heatsink, remove the protective film covering the thermal interface material on the bottom side of the heat sink; no grease is needed for this situation.)
15.
Attach the heatsink. Position the heatsink fins in the proper orientation.
a. Start with screw A and engage the screw threads by giving it two rotations. (Do not fully tighten.) b. Proceed to screw B and engage screw threads by giving it two rotations. Continue by engaging screws C and D.
c. Continue by giving each screw, using the same pattern, two rotations each time until each screw is lightly tightened. Torque each screw to 8 inch-lb.
CAUTION:
● The heatsinks for processor 1 and processor 2 are different.
● S2600 motherboards use two different heatsinks for processors 1 and 2. The two heatsinks are not interchangeable. Using the wrong heatsink will cause serious thermal damage. A heatsink
(and socket cover) is required even if no processor is installed.
HR90-2009 CS400-AC Hardware Replacement Procedures 38
Motherboard, Processor and DIMM Replacement
DIMM Replacement
Prerequisites
● Part Numbers:
○ 100944700 - DIMM, 8GB DDR4-2133 Reg dual rank 2RX8
○ 100961800 - DIMM, 16GB DDR4-2133 Reg dual rank ECC 1.2V 1GX4
○ 101107700 - DIMM, 32GB DDR4-2133 Reg dual rank ECC 1.2V 1GX4
About this task
This procedure describes how to replace DIMM modules.
Motherboards are not shipped with processors or DIMMs installed. Processors, heatsinks, and DIMMs must be removed from the defective motherboard and installed on the replacement motherboard.
Remove DIMM
1.
Press down on the DIMM latches to unseat the DIMM from the DIMM socket. The DIMM lifts from the socket.
2.
Holding the DIMM by the edges, lift it straight up from the socket and set the DIMM on an ESD-safe work surface. If the DIMM will not be reinstalled soon, store it in an anti-static package.
HR90-2009 CS400-AC Hardware Replacement Procedures 39
Motherboard, Processor and DIMM Replacement
Install DIMM
3.
Make sure the clips at either end of the DIMM socket are pushed outward to the open position.
4.
Holding the DIMM by the edges, remove it from its anti-static package. Position the DIMM above the socket.
Align the notch on the bottom edge of the DIMM with the key in the DIMM socket.
5.
Insert the bottom edge of the DIMM into the socket (A). When the DIMM is inserted, push down on the top edge of the DIMM until the retaining clips snap into place (B). Make sure the clips are firmly in place (C).
HR90-2009 CS400-AC Hardware Replacement Procedures 40
Subrack Parts Replacement Procedures
Subrack Parts Replacement Procedures
The following procedures are included in this section:
● CS400-AC Fan Assembly Replacement
● iSCB Module PCBA Replacement
● SR5110 Midplane and Power Backplane Replacement
● CS400 Subrack Support Angles
● SR Subrack Replacement
● SR Subrack Installation
CS400-AC Fan Assembly Replacement
Prerequisites
● 002-00372B - Fan module
● 22302-0270-01 - Fan interface board
● 23101-0295-01 - Dual-fan assembly with wire harness
About this task
The figure below provides an exploded view of the fan module to show how all the parts and replacement components fit together. Refer to the following figure for replacing any parts of a fan module.
HR90-2009 CS400-AC Hardware Replacement Procedures 41
Subrack Parts Replacement Procedures
iSCB Module PCBA Replacement (CS400-AC)
Prerequisites
● 002-00374C - iSCB module
● 22901-0225-00 - iSCB serial port cable assembly
● 22302-0313-01 - PCBA I/O network board
● 22309-00005-04 - PCBA CPU board
About this task
This procedure describes how to replace either of the two printed circuit board assemblies (PCBAs) in an iSCB module. Reverse the process to replace the PCBAs.
Note: The two PCBAs are connected by a 28-pin connector as shown in the figure below. The CPU PCBA must be removed first, before the I/O, network PCBA can be removed.
HR90-2009 CS400-AC Hardware Replacement Procedures 42
Subrack Parts Replacement Procedures
Procedure
iSCB CPU PCBA Removal
1.
Disconnect the 9-pin serial port (ribbon cable) connector from the CPU PCBA.
2.
Remove the 4 screws from the CPU PCBA.
3.
Pull the PCBA to disengage the 28-pin connectors between the two PCBAs.
iSCB I/O, Network PCBA Removal
4.
Peel one side of the film tape that secures the ribbon cable to the PCBA. Place the ribbon cable to the side of the iSCB module.
5.
Remove the 3 screws from the I/O, network PCBA.
Midplane edge connectors
Plugs into 9-pin connector on CPU board
Ribbon cable may be taped to I/O network board
Serial port cable assembly
(male DB-9 to 10-pin)
22901-0225-00
Pan Screws
(6-32 X 5/16)
PCBA - CPU board
(22309-0005-04)
28-pin connectors
PCBA - I/O network board
(22302-0313-01)
COM
GreenBlade
002-00374C - iSCB Module
(SR5110/SR8204/SR8116)
6.
Remove the I/O, network PCBA. Remove the piece of film tape and use it to secure the ribbon cable to the replacement PCBA.
HR90-2009 CS400-AC Hardware Replacement Procedures 43
Subrack Parts Replacement Procedures
SR5110 Midplane and Power Backplane Replacement
About this task
This procedure describes how to access and replace the midplane and power backplane in an SR5110 subrack.
Refer to the following exploded view of the SR5110 that shows screw locations and the intended assembly of parts.
NODE
NODE
Procedure
1.
Power off all blades and systems in the subrack.
HR90-2009 CS400-AC Hardware Replacement Procedures 44
Subrack Parts Replacement Procedures
2.
Remove the subrack from the chassis and place on an ESD safe work surface.
3.
Unseat all GB522X blades, power supplies, and the iSCB module.
Pull each about 3 inches out of the rack. (All modules/supplies can remain in the SR5110 subrack.)
4.
Remove the three fan modules (3).
5.
Remove the front bezel (4).
6.
Remove the front panel logo assembly (6).
a. Remove three screws (11) that secure the assembly to the fan cage.
b. Use a long-shaft Phillips screwdriver to loosen the captive screw (7) on each side of the assembly.
c. Slowly pull the assembly straight out to unseat the front panel connectors from the midplane.
7.
Remove the fan cage assembly (1). [The handles on the subrack are attached to the fan cage assembly. Do not remove the handles.] a. Remove the four screws (20) from each side of the subrack. These screws secure the fan cage assembly to the subrack chassis.
b. Remove the two screws (19) from the top of the subrack.
c. Gently pull the fan cage assembly straight out of the front of the subrack [The connectors on the fan
PCBA module will unseat from the midplane.]
Remove the power backplane and midplane as one unit
To access/replace the midplane or power backplane, it is easier to remove the power backplane plate (with attached power PCBA and air duct) and midplane as a single unit. There are 9 screws in the midplane. The screw in the bottom middle of the midplane attaches the midplane to a standoff on the power backplane plate. This process is described in the following steps.
8.
Remove the 6 screws along the top of the midplane (12).
9.
Remove the screw on each end of the midplane. [Each screw attaches to a separate bracket (14) on the inside of the chassis.]
10.
Loosen the captive screws (13) that attach to brackets (16) in the base of the chassis.
11.
Pull the power backplane plate with attached midplane straight out the front of the subrack.
This entire assembly can now be placed on a work surface and inspected/viewed from any angle.
Replace the power backplane
The power backplane is screwed to the bottom of the power backplane plate. Replacement backplanes may not include an air duct. The air duct from the failed backplane will need to be removed and attached to the replacement backplane.
12.
To remove the air duct, first carefully remove the two black plastic rivets at the center of the air duct.
13.
Then remove the two screws.
14.
Remove the screws that attach the backplane PCBA to the plate. DO NOT remove the heatsink screws.
HR90-2009 CS400-AC Hardware Replacement Procedures 45
Subrack Parts Replacement Procedures
15.
Attach the new backplane PCBA. Attach the air duct. Then reassemble all parts by following the above steps in reverse order.
Replace the midplane
The midplane is attached to the power backplane plate through a single standoff. One screw.
16.
Attach the new midplane. Then reassemble all parts by following the above steps in reverse order.
CS400 Subrack Support Angles
Each CS400 Subrack (SR5110/SR6110/SR8204/SR8116) sits on a pair of support brackets mounted to the rack.
These support brackets are included in all preconfigured Cray® CS400™ cluster supercomputer cabinets. A set of support bracket/angle assemblies (left and right), a cable management bar, screws/nuts, and tie wrap is included with new/replacement SR subracks.
Support bracket/angle part numbers for 19-inch racks:
● 101243300: left support angle assembly
● 101243400: right support angle assembly
HR90-2009 CS400-AC Hardware Replacement Procedures 46
Subrack Parts Replacement Procedures
SR Subrack Replacement
About this task
This procedure describes how to remove and replace an entire SR5110 subrack in a CS400-AC cabinet.
Components and assemblies can be removed from the rear of the failed subrack before it is removed from the
CS400-AC cabinet. Procedures for SR8204 and SR8116 subracks are nearly the same so separate procedures for these subracks are not provided.
GreenBlade SR subrack chassis sit on a pair of support angles that are mounted to the rack. The support angles create a shelf. The subrack is secured to the vertical rails of the rack with two screws through the mounting holes of each of the subrack handle brackets.
CAUTION:
● Serious injury or equipment damage
● A fully populated SR subrack chassis can weigh up to 273 lb (124 kg). A lift is required to safely install and remove these subracks from a rack. Failure to do so may result in serious injury or equipment damage.
Removal
1.
Unpack the replacement subrack.
HR90-2009 CS400-AC Hardware Replacement Procedures 47
Subrack Parts Replacement Procedures
2.
Power off all blades/systems in the failed subrack
3.
Open the front and rear doors of the CS400-AC cabinet.
Subrack Component Removal - Rear
4.
Disconnect all power cords and data cables connected to the failed subrack.
5.
Place the cable management bar in the down position.
Depending on the configuration, the cable management bar (four M6 screws) may need to be removed to provide clearance to remove the iSCB module and power supplies.
6.
Remove the iSCB module.
7.
Remove the PSUs.
8.
Remove all GB522X blades.
Subrack Removal - Front
9.
Remove the front bezel.
10.
Write down the positions of the three DIP switches. This will make it easier to properly set the switches in the replacement subrack.
11.
Remove the screws mounting the subrack handles to the rack frame. (Four M6 screws.)
12.
Grasp the subrack handles and carefully pull the subrack out approximately 2 inches from the cabinet.
13.
Position the lift in front of the rack and raise the platform so it is flush with the bottom of the subrack.
14.
Set the lift brake.
HR90-2009 CS400-AC Hardware Replacement Procedures 48
Subrack Parts Replacement Procedures
15.
Carefully slide the server out of the rack and onto the lift platform.
16. IMPORTANT: If the replacement subrack will not be installed within a few minutes, install a blank filler panel(s). Failure to do so can disrupt airflow and overheat other components in the rack.
Install a blank panel(s) in the opening and secure the panel to the rack. If necessary, remove the plastic fasteners so that screws can be used to secure the filler panel to the rack.
SR Subrack Installation
About this task
This procedure describes how to install a new or replacement SR5110 subrack.
Install Subrack in Rack
1.
Carefully slide the subrack off the work surface and onto the lift platform.
2.
Position the lift in front of the rack and set the brake.
3.
Remove any blank filler panel(s).
4.
Carefully slide the subrack off the lift platform and onto the support angles in the opening in the rack until the front panel is flush with the rack rails.
5.
Release the brake, and slowly move the lift away from the rack.
6.
Secure the subrack to the rack frame with four M6 screws.
7.
Remove the front bezel and set the DIP switches.
Install Modules in Subrack
8.
Install the cable management bar and place it in the down position
9.
Install modules:
● PSUs
● iSCB module
● GB522X modules
10.
Connect all power cords and data cables. Refer to the numbers on the cable labels.
11.
Check all connections before powering up the system.
HR90-2009 CS400-AC Hardware Replacement Procedures 49

Public link updated
The public link to your chat has been updated.
Advertisement