User Manual

216 Port InfiniBand FDR Switch Platform

Hardware User Manual

PN:MSX6512-4R. MSX6512-NR

Rev. 1.5

www.mellanox.com

Rev 1.5

NOTE:

THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED

DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY

KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE

THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT

HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE

PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND

DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST

QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT

ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES

FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND

(INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;

LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY

THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE

OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED

DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Mellanox Technologies

350 Oakmead Parkway Suite 100

Sunnyvale, CA 94085

U.S.A.

www.mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies, Ltd.

Beit Mellanox

PO Box 586 Yokneam 20692

Israel www.mellanox.com

Tel: +972 (0)74 723 7200

Fax: +972 (0)4 959 3245

© Copyright 2012. Mellanox Technologies. All Rights Reserved.

Mellanox®, Mellanox logo, BridgeX®, ConnectX®, CORE-Direct®, InfiniBridge®, InfiniHost®, InfiniScale®, PhyX®,

SwitchX®, Virtual Protocol Interconnect® and Voltaire® are registered trademarks of Mellanox Technologies, Ltd.

Connect-IB™, FabricIT™, MLNX-OS™, ScalableHPC™, Unbreakable-Link™, UFM™ and Unified Fabric Manager™ are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners.

2 Mellanox Technologies Document Number: 3835

InfiniBand FDR Switch Platform Hardware User Manual

Table of Contents

Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Mellanox Part Numbering Legend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Chapter 1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.1 Product Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.1.1 Serial Number and Product Version Information . . . . . . . . . . . . . . . . . . . 12

1.1.2 Management Module MAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.1.3 Product Physical Specifications and Power . . . . . . . . . . . . . . . . . . . . . . . 13

1.2 Features List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.3 InfiniBand FDR and FDR10 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Chapter 2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1 Installation Safety Warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 Power Supply Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2.1 1000W Power Supply Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2.2 1600W Power Supply Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3 LED Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.1 Power Supply Unit LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.2 Leaf Board LED Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.3.3 Spine Board LED Indicators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.4 Spine Side Panel Display LED Indicators . . . . . . . . . . . . . . . . . . . . . . . . 27

2.3.5 Management Module LED Indicators. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.3.6 Port Connector Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.4 Air Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.5 QSFP Cable Power Budget Classification. . . . . . . . . . . . . . . . . . . . . . . . 30

2.6 Management Module Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.6.1 I2C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.6.2 CONSOLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.6.3 MGT– Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.6.4 USB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.6.5 Reset – RST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Chapter 3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1 Environmental and Safety Recommendations . . . . . . . . . . . . . . . . . . . . 33

3.2 Chassis Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.3 Leaf Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.4 Spine Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.5 Management Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6 Physical Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.6.1 ESD Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.6.2 Installation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.6.3 Installing the Cable Holder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42


Rev 1.5

3

Rev 1.5

3.6.4 Installing the Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.6.5 Ground Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.7 Power Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.7.1 Powering Up the Switch Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.8 InfiniBand QSFP Cable Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.8.1 Supported Approved Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.8.2 Cable Power Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Chapter 4 Chassis Power Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.1 Power Supply and Spine Board Indicator Status at Power ON . . . . . . . . 56

Chapter 5 Insertion and/or Hot swap Extraction. . . . . . . . . . . . . . . . . . . . . . . . 58

5.1 Power Supply Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.1.1 Extracting and Inserting the Power Supply Unit . . . . . . . . . . . . . . . . . . . . 58

5.2 Leaf Boards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.2.1 Extracting a Leaf Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.2.2 Inserting a Leaf Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.3 Spine Boards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.3.1 Extracting a Spine Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.3.2 Inserting a Spine Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.4 Fan Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.4.1 Leaf Fan Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.4.2 Spine Fan Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

5.5 Management Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.5.1 Extracting a Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.5.2 Inserting a Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5.6 Switch Shut Down Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Chapter 6 Disassembly and Disposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.1 Disassembling the Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.1.1 Removing the Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.1.2 Removing the Bottom Shelf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6.2 Disposal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Chapter 7 Switch Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

7.1 InfiniBand Subnet Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

7.2 Fabric Inspector (Diagnostics). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

7.3 Accessing the CPU via the Ethernet Connector . . . . . . . . . . . . . . . . . . . 73

7.4 Upgrading Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Chapter 8 TroubleShooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8.1 Power Supply Unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8.2 Leaf Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8.3 Management Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

8.3.1 Yellow Status LED for the Chassis on the Management Module is Lit. . . 75

8.3.2 Yellow LED for the Leaf Fan on the Management Module is Lit . . . . . . . 75

8.3.3 Yellow LED for the Spine Fan on the Management Module is Lit . . . . . . 76

8.4 Spine Board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

8.5 MLNX-OS™ Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Appendix A Specification Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

A.1 EMI Certification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4 Mellanox Technologies


A.2 Approved Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

A.3 EMC Certifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Appendix B Thermal Threshold Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Appendix C Calculating the Weight of a Customized Chassis . . . . . . . . . . . . 81

Appendix D Calculating the Power of a Chassis . . . . . . . . . . . . . . . . . . . . . . . 82

Appendix E QSFP Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Appendix F Replacement Parts Ordering Numbers . . . . . . . . . . . . . . . . . . . . . 85

Appendix G Avertissements de sécurité d’installation (French) . . . . . . . . . . 87

Appendix H Installation - Sicherheitshinweise (German) . . . . . . . . . . . . . . . . 91

Appendix I Advertencias de seguridad para la instalación (Spanish) . . . . . . 95

Rev 1.5

Mellanox Technologies 5

Rev 1.5

List of Figures

Figure 1: Generic Product Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

Figure 2: Management Module MAC Address Location . . . . . . . . . . . . . . . . . . . . . . . . . .13

Figure 3: SX6512 Switch Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Figure 4: Power Supply Unit Status Indications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

Figure 5: PSU Cover On and Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

Figure 6: Leaf Board Led Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

Figure 7: Spine Status LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Figure 8: Spine Side Panel Display Status Indications . . . . . . . . . . . . . . . . . . . . . . . . . . .28

Figure 9: Management Module Status Indications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Figure 10: Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Figure 11: Top and Bottom Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Figure 12: Management Module Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Figure 13: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Figure 14: Shelf Installation Kit Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Figure 15: Chassis Installation Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Figure 16: Shock and Tilt Stickers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Figure 17: Inserting the Caged Nuts for the Shelf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Figure 18: Connect Rail Slide to Rack Vertical support . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Figure 19: Inserting the Caged Nuts for the Faceplate . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

Figure 20: Cable Holders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Figure 21: Cable Holder Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Figure 22: Chassis Rails and Rail Slides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45

Figure 23: Placement of Chassis in Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46

Figure 24: Installing the Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46

Figure 25: Screw the Handles Onto the Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47

Figure 26: The Rails are Already Connected Onto the Chassis . . . . . . . . . . . . . . . . . . . . .48

Figure 27: Raising the Chassis Using a Mechanical Lift . . . . . . . . . . . . . . . . . . . . . . . . . . .49

Figure 28: Put on the Rail Slide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

Figure 29: SX6512 Chassis on the Shelf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50

Figure 30: Face Plate Mounting Bolt Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50

Figure 31: Move the Rail Slide to the Vertical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

Figure 32: Ground Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52

Figure 33: Multiple Power Inlets - Electric Caution Notification . . . . . . . . . . . . . . . . . . . . . .53

Figure 34: Spine Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54

Figure 35: Spine Side Panel Display Status Indications . . . . . . . . . . . . . . . . . . . . . . . . . . .57

Figure 36: Management Module Status Indications for Normal Operation . . . . . . . . . . . . .57

Figure 37: Power Cord Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59

Figure 38: PSU Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59



Figure 39: Power Supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Figure 40: Leaf Board Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Figure 41: Ejector Latch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Figure 42: Leaf Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Figure 43: Management Module Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63


Figure 45: Spine Board Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Figure 46: Fan Locations on the Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Figure 47: Leaf Fan Module Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Figure 48: Spine Fan Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Figure 49: Fan Status LED on the Spine Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66


Figure 51: Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Figure 52: QSFP Connector Male and Female Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Rev 1.5


8

Rev 1.5

List of Tables

Table 1: Revision History of this User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Table 2: Reference Documents and Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Table 3: Switch Rack Mechanical and Environmental Requirements . . . . . . . . . . . . . . . .13

Table 4: OPNs for Power Supply Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

Table 5: Leaf Status LED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

Table 6: Bad Port LED Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

Table 7: Connector Physical and Logical Link Indications . . . . . . . . . . . . . . . . . . . . . . . .25

Table 8: Spine Status LED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Table 9: Spine Fan Status LED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Table 10: Spine to Leaf IB Link Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

Table 11: Bad Port LED Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

Table 12: LEDs Display for Normal Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28

Table 13: LEDs Display for Normal Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Table 14: Switch Specification Sheet SX6512-[4N]R . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

Table 15: Power Consumption of Chassis Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82

Table 16: InfiniBand QSFP Connector Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83

Table 17: Replacement Parts Ordering Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85


Rev 1.5


Revision History

Table 1 - Revision History of this User’s Manual

Revision

1.5

1.4

1.3

1.2

1.1

1.0

Date Details

October

2012

Added Warning for handles

September

2012

Added figure 23.

Added information regarding inserting the bottom spine first. Insert spines from bottom to top.

August,

2012

Minor changes due to changes in container.

July, 2012 Fixed broken link.

July, 2012 Added figure to show use of 2nd type of mechanical lift

Added EMI explanation to Appendix

Added explanation to N+2 Power Numbers.

May, 2012 Initial release

About this Manual

This manual provides an overview of the SwitchX® based SX6512 modular InfiniBand chassis switch, and guidelines for its operation.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting up the chassis platform.

The manual assumes familiarity with the InfiniBand

®

architecture specification.

Related Documentation

The documentation set accompanying the QSFP Chassis InfiniBand Switch platform includes the following:

Table 2 - Reference Documents and Web Sites

Document Name

InfiniBand Architecture Specification, Vol. 1, Release 1.2.1

Switch Hardware Errata

Description

The InfiniBand Architecture Specification that is provided by IBTA

For any possible errata due to hardware issues see the switch support product page. This requires a customer support login.


Rev 1.5

Table 2 - Reference Documents and Web Sites

Document Name Description

Mellanox MLNX-OS™ SwitchX®

Software WebUI User’s Manual

Mellanox MLNX-OS™ SwitchX®

Software User Manual

WebUI Overview for MLNX-OS software.

This document contains information regarding configuring and managing Mellanox Technologies' SwitchX Switch Platforms.

MLNX-OS™ Software Command Reference Guide

Command Reference Guide for MLNX-OS listing all of the commands available through MLNX-OS with explanations and examples.

MLNX-OS™ Software Configuration

Guide

Configuration Guide for MLNX-OS displaying different configuration scenarios.

Conventions

Throughout this manual, the name SX6512 and the terms chassis and switch are used to describe the 216 port QSFP InfiniBand chassis, unless explicitly indicated otherwise.

The following icons are used throughout this document to indicate information that is important to the user.

This symbol makes recommendations to the user.

This symbol indicates information that is helpful to the user.

This symbol indicates a situation that can potentially cause damage to hardware or software.

Warning! This symbol indicates a situation that can potentially cause personal injury and / or damage to hardware or software.



Mellanox Part Numbering Legend

-

P

G

FF

M

SX

PR

R

Place Field Decoder


System Type SwitchX Switch

Data Transfer Protocol (1, 2, 3, 4) = Ethernet (5, 6, 7, 8) = InfiniBand

Module Generation

Number of leafs

Separator

# Power Supplies

5, 6, 7, 8

36, 18, 12, 06

Chip Generation

N = N+N redundant

0=0, 1=1, 2=2....

R – SwitchX

S – SwitchX-2

Rev 1.5


Rev 1.5

1 Overview

This User Manual provides an overview of the SX6512 QSFP Modular InfiniBand Switch Platform (known in this document as ‘the chassis or switch’) and its operational environment.

Mellanox SX6512 switch systems provide the highest performing fabric solution by delivering high bandwidth and low latency to Enterprise Data Centers (EDC), High-Performance Computing (HPC) and Embedded environments. Networks built with the SX6512 system can carry converged traffic with the combination of assured bandwidth and granular quality of service. Built with Mellanox’s 5 th

generation SwitchX® VPI switch device, SX6512 systems provide up to

56Gb/s full bidirectional bandwidth per port. With up to 216 ports in a 9U high form factor, these systems are among the densest switching systems available.

The switch platform comes pre-installed with all necessary firmware for standard operation within an InfiniBand fabric and requires an InfiniBand compliant Subnet Manager running from one of the hosts or the management module of the switch system. An initial configuration procedure should be followed to initialize the switch before connecting it to the network after which normal operation can proceed. (See the installation guide for details regarding the initial configuration.) Once connected to the network, the Subnet Management software automatically discovers and configures the fabric and begins utilizing the switch.

The Mellanox Operating System (MLNX-OS™) software package provides a subnet manager and network management tools as well as connectivity software for servers and storage, and is available on the Mellanox web site.

Basic installation is covered in

Chapter 3, “Installation” on page 33

.

Hot-swapping components and hardware maintenance is covered in

Chapter 5, “Insertion and/ or Hot swap Extraction” on page 58 .

1.1

Product Information

Overview

1.1.1

Serial Number and Product Version Information

The serial number, GUID Identifier and product version information are found on the label that is attached to the pull-out tab that is below the Mellanox logo on the spine side of the chassis.

Figure 1: Generic Product Label

S / N : M T 0 8 0 6 X 0 1 5 0 4

P / N : M S X 6 5 1 2 ‐ 4 R

G U ID : 0 0 0 2 C 9 0 2 0 0 2 7 0 5 1 C

M A C : 0 0 0 2 C 9 0 2 0 0 4 0

M a d e in Is r a e l

R e v :A 1

The GUID is the System Image GUID according to the IB spec. It is burned on the board which is in the chassis. All the boards and the management software look for this GUID in addition to their own Node GUID.



1.1.2

Management Module MAC

Rev 1.5

Here is the tab with the product label n #3 Fan #4

SX6512

R

Each management module has a label with its MAC address. See below for the location of this label.

IB

STATUS

Figure 2: Management Module MAC Address Location

I2C CONSOLE MGT

RST PSU S.Fans

L.Fans

Master

MAC address location

1.1.3

Product Physical Specifications and Power

Table 3 - Switch Rack Mechanical and Environmental Requirements

Rack

Height

Required

Rack Width

Rack

Depth

# of leafs

/

# of Spines

10 U

(444mm) including bottom shelf

19”

(EIA-310)

(482.6 mm)

25.7”



685 mm

12/6

Weight Max. Power (216 x QSFP Ports)

Ambient

Temp.

120 kg (265 LBS)

Full configuration

48 kg (105 LBS)

Empty configuration

83.94 kg (185 LBS)

Shipped configuration

Typical: 2553.59W (including

QSFP at 2W)

Typical: 2337.59W (including

QSFP at 1W)

Max: 2815.11W (including

QSFP at 2W)

Up to 4 Power Supplies of 100-

240VAC, 50-60Hz, 6.3-13A

Max: 45ºC

Min: 0ºC

The switch ships in a minimum base configuration plus additional modules depending on the chosen customer configuration. Optional modules included:

• Leaf boards

• Management modules

• Spine boards

Figure 3 shows the connector and spine sides fully populated.


Rev 1.5

Figure 3: SX6512 Switch Views

AC1 AC2 AC3

F

D

R

UID

F

D

R

UID

F

D

R

UID

F

D

R

F

D

R

UID

F

D

R

UID

UID

IB

STATUS

I2C

IB

STATUS

I2C

CONSOLE MGT

CONSOLE MGT

RST PSU S.Fans

L.Fans

Master

RST PSU S.Fans

L.Fans

Master

F

D

R

UID

F

D

R

UID

F

D

R

F

D

R

UID

D

F

R

UID

F

D

R

UID

UID

AC4

Spine Side

Mellanox

®

L

PS1

PS3

Fan #1

S1

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

Fan #2 Fan #3 Fan #4

SX6512

R

PS2

PS4

Up to 4 power supply units

6 spine modules

4 leaf fan modules

12 leaf modules

2 management modules

Overview



1.2

Features List

• 216 FDR (56Gb/s) InfiniBand ports in a 9U switch

• 24.2 Tb/s aggregate data switching capacity with ultra low latency

• IBTA 1.3 and 1.21 compliant

• SDR/DDR/QDR/FDR10/FDR link speed

• Congestion control

1

• Adaptive routing

1

• Port mirroring

1

• Chassis High Availability

• sMB High Availability

1.3

InfiniBand FDR and FDR10 Overview

The Mellanox SX6512 switch system supports FDR, a pre-standard InfiniBand data rate, where each lane of a 4X port runs a bit rate of 14.0625Gb/s with a 64b/66b encoding, resulting in an effective bandwidth of 54.54Gb/s. The FDR physical layer is an IBTA specified physical layer using different block types, deskew mechanism and framing rules.

The SX6512 switch also supports FDR10, a non-standard InfiniBand data rate, where each lane of a 4X port runs a bit rate of 10.3125Gb/s with a 64b/66b encoding, resulting in an effective bandwidth of 40Gb/s.

FDR10 supports 20% more bandwidth over regular QDR using the same QSFP cables/ connectors.

Both FDR and FDR10 support Forward Error Correction (FEC), as described in IEEE

Std 802.3ap™-2007 (Amendment to IEEE Std 802.3™-2005) chapter 74.

Rev 1.5

FDR and FDR10 are only guaranteed to work with approved Mellanox Cables.

FDR10 is only guaranteed to work with approved Mellanox ConnectX-3 adapters.

1. Features for a future release.


Rev 1.5

2 Hardware

2.1

Installation Safety Warnings

These safety warnings are in English. For French, Spanish, and German see the Appendixes.

1. Installation Instructions

Hardware

Read all installation instructions before connecting the equipment to the power source.

2. Bodily Injury Due to Weight

Use enough people to safely lift this product.

16

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

3. Installation in Restricted Access Location.

70 - 121 lbs

32 - 55 kgs

This unit is intended for installation in a Restricted Access Location.

>121 lbs

>55 kgs

4. Over-temperature

This equipment should not be operated in an area with an ambient temperature exceeding the maximum recommended: 45°C (113°F). Moreover, to guarantee proper air flow, allow at least 8cm (3 inches) of clearance around the ventilation openings.

5. Stacking the Chassis

The chassis should not be stacked on any other equipment. If the chassis falls, it can cause bodily injury and equipment damage.

6. Redundant Power Supply Connection - Electrical Hazard

This product includes a redundant power or a blank in its place. In case of a blank power supply, do not operate the product with the blank cover removed or not securely fastened.



7. Multiple Power Inlets

Risk of electric shock and energy hazard.

The PSUs are all independent.

Disconnect all power supplies to ensure a powered down state inside of the switch platform.

8. During Lightning - Electrical Hazard

During periods of lightning activity, do not work on the equipment or connect or disconnect cables.

9. Copper InfiniBand Cable Connecting/Disconnecting

Copper InfiniBand cables are heavy and not flexible, as such they should be carefully attached to or detached from the connectors. Refer to the cable manufacturer for special warnings/instructions.

10. Rack Mounting and Servicing

When this product is mounted or serviced in a rack, special precautions must be taken to ensure that the system remains stable. In general you should fill the rack with equipment starting from the bottom to the top.

11. Equipment Installation

This equipment should be installed, replaced, and/or serviced only by trained and qualified personnel.

12. Proper Enclosure

A suitable electrical, mechanical and fire enclosure shall be provided by the end product manufacturer and or the end user.

13. Equipment Disposal

Disposal of this equipment should be in accordance to all national laws and regulations.

Rev 1.5


Rev 1.5

18

Hardware

14. Local and National Electrical Codes

This equipment should be installed in compliance with local and national electrical codes.

15. UL Listed and CSA Certified Power Supply Cord Leakage >3.5mA

For North American power connection, select a power supply cord that is UL Listed and CSA Certified, 3 - conductor, [16 AWG], terminated with a molded plug rated at

125 V, [13 A], with a minimum length of 1.5m [six feet] but no longer than 4.5m.

For European connection, select a power supply cord that is internationally

 harmonized and marked “<HAR>”, 3 - conductor, minimum 1.0 mm

2

wire, rated at 300 V, with a PVC insulated jacket. The cord must have a molded plug rated at 250 V, 10 A.

16. High Leakage Current

Warning: High leakage current; Earth connection essential before connecting supply.

17. Add GND connection information

Before connecting this device to the power line, the protective earth terminal screws of this device must be connected to the protective earth in the building installation.

(GND Connection Information):

The building installation shall provide a means for a connection to protective earth; and the equipment shall be permanently connected to that by a service person.

A SERVICE PERSON shall check whether or not the socket - outlet from which the equipment is to be powered provides a connection to the building protective earth. If not, the SERVICE PERSON shall arrange for the installation of a PROTECTIVE

EARTHING CONDUCTOR from the separate protective earthing terminal to the protective earth wire in the building. The equipment shall be installed in area where equipotential bonding exists ((such as a telecommunication centre or a dedicated computer room).

18. Installation codes

This device must be installed according to the latest version of the country national electrical codes. For North America, equipment must be installed in accordance to the applicable requirements in the US National Electrical Code and the Canadian Electrical Code.

19. Interconnection of units

Cables for connecting to the unit RS232 and Ethernet Interfaces must be UL certified type DP-1 or DP-2. (Note- when residing in non LPS circuit)

Overcurrent Protection: A readily accessible Listed branch circuit overcurrent protective device rated 20 A must be incorporated in the building wiring.



20. Hazardous Radiation Exposure

Caution – Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure.

Rev 1.5

CLASS 1 LASER PRODUCT and reference to the most recent laser standards:



IEC 60825-1:2007/03 and EN 60825-1:2007

21. Do Not Use the Switch as a Shelf or Work Space

Caution: Slide/rail mounted equipment is not to be used as a shelf or a work space.

22. WEEE Directive

According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not disposed of with regular household waste.

Dispose of this product and all of its parts in a responsible and environmentally friendly way.

23. Country of Norway Power Restrictions

This unit is intended for connection to a TN power system and an IT power system of

Norway only.


Rev 1.5

Hardware

2.2

Power Supply Redundancy

The MSX6512 platform comes standard with four power supplies and has two order options:

• MSX6512-4R - N+1 1000W output per PSU

• MSX6512-NR - N+N 1600W output per PSU

2.2.1

1000W Power Supply Units

The 1000 W power supplies deliver 1000W at 48V. The input to each of these power supplies requires 1214.46W at 220V in order to output 1000W.

With 1000W PSUs the only redundancy option is N+1. In this configuration 1PSU (Power Supply Unit) can be extracted without bringing down the system. When using the 1000W PSUs, the minimum complement of power supply units is 4 and the chassis will continue to run at full capacity with only 3 PSUs.

2.2.2

1600W Power Supply Units

When using the 1600W PSUs, the minimum complement of power supply units is 2 and the chassis will continue to run at full capacity with only 2 PSUs.

The 1600W PSUs can implement two redundancy options.

• N+1 configuration

The chassis when supplied by a single power grid can be configured for N+1 times 2. This means that all PSUs are supplied from a single power source usually the commercial electric grid. This enables 2 power supply redundancy.

• N+N configuration

N+N with 1600W PSUs will allow for N+N redundancy. This requires a second power grid supplied by a

• backup power supply grid

• generator

• battery backup system

• any combination of the above

Connecting 2 power supplies to one power supply grid and the remaining 2 power supplies to a secondary power supply grid will create N+N redundancy. This is High Availability. Under these conditions should a power supply fail (an electric company power failure or blackout for example) power grid High Availability will continue to keep the chassis running at full capacity through the secondary or backup power supply grid.

With N+N optional PSU grid redundancy the chassis can run on 1/2 of the full complement of

PSUs. N+N allows the chassis to run on 2 PSUs supplied from one power grid while 2 are connected to a second power grid.

With power grid A charged with current and power grid B not charged there is only grid redundancy and not PSU redundancy.

20

MLNX-OS™ may power down some leafs. If this happens it may be necessary to reboot the chassis once the defective PSU has been replaced. Two simple ways to reboot is to use the reboot command in the CLI or reboot through the Web GUI.



The form is identical between the 1600W and the 1000W PSUs. The standard PSU weighs 0.3kg more than the 1600W PSU.

N+N redundancy ONLY works with a supply voltage of 220V.

Table 4 - OPNs for Power Supply Units

OPN

PSU

Wattage

Description

MTP005001 1000 W

Supplies N+2 redundancy for

SX6536 and SX6518 chassis

Supplies N+1 redundancy for

SX6512 and SX6506 chassis

MTP006002 1600 W Supplies N+N redundancy for all switch chassis at 220 Volts

Rev 1.5


Rev 1.5

2.3

LED Status Indicators

The LEDs are placed on the chassis for the convenience of the IT manager. All chassis conditions and management options are available and controllable through the management SW, either CLI or Web GUI.

Hardware

It is recommended that all of the chassis sub systems be maintained and managed through the management software.

2.3.1

Power Supply Unit LEDs

Each Power Supply Unit has the following indicator LEDs.

Figure 4: Power Supply Unit Status Indications

DC

AC

OK FAIL

OK

AC – When lit this LED indicates input voltage between 100 and 240 Volts.

DC FAIL – When lit this LED indicates a fault in the power supply.

DC OK – When lit this LED indicates that the output from the power supply is +48 VDC.

The PSUs are on the spine side of the chassis behind two cover panels. The plugs for these PSUs are on the leaf side of the chassis.



Figure 5: PSU Cover On and Off

PSU Covers

Mellanox

®

L

PS1

PS3

Fan #1

S1

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3

5

7 9 11 13

15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13

15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18


SX6512

R

PS2

PS4

PSU Covers

Mellanox

®

Fan #1

S1

1 3

5 7

9 11 13

15 17

UID 2 4 6 8 10 12 14 16 18

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8

10

12

14 16

18

1 3

5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13

15 17

UID 2 4 6 8 10 12 14 16 18


SX6512

R

Rev 1.5


Rev 1.5

2.3.2

Leaf Board LED Indicators

Figure 6: Leaf Board Led Indicators

F

D

R

1

2

3

4

5

6

7

8

9

10

UID

Hardware

2.3.2.1 Status LED

Each leaf board has a Status LED on the far left of the leaf. Table 5 shows the leaf status accord-

ing to the LED condition.

The Leaf Status indicator LED has the following LED assignment:

Table 5 - Leaf Status LED

LED Condition

Off

Solid Green

Flashing Green

Solid Orange

Solid Red

LED Description

No power to the Leaf

Leaf is up and running

Leaf is powering up

Non fatal error – this leaf needs troubleshooting, but does not require chassis shutdown

Fatal error

2.3.2.2 Bad Port LED

The Bad Port indicator is located on the left side of the leaf. The following Bad Port conditions are possible:

Table 6 - Bad Port LED Configurations

LED Configuration Description

Off OK – all ports are up and running.

Flashing Orange Error – one or possibly more ports has just received a symbol error

This LED shows symbol errors. Possible causes for this are:

• bad cable

• bad connection

• bad connector

This LED lights up when one or more ports is receiving a symbol error. The LED immediately goes off until the next symbol error is received.



2.3.2.3 UID LED Switch Identifier

The UID LED is a debug feature that will become available to customers in the near future. For details please contact Mellanox Technologies support.

2.3.2.4 Leaf Board Port Connector LED Assignment

Above the ports are two LEDs one for the upper port and one for the lower port .

Each port has a single 2 color LED. Table 7 shows the link status according to the LED

condition.

Table 7 - Connector Physical and Logical Link Indications

LED Condition

Off

Solid Green

Flashing Green

Solid Orange

Flashing Orange

LED Description

No power to the port

Logical link up

Data activity



– flashing speed is proportional to data transfer speed

Physical link up

A problem with the physical link

The LED indicator, corresponding to each data port, will light orange when the physical connection is established (that is, when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port). When a logical connection is made the LED will change to green. When data is being transferred the light will blink green.

The switch does not provide a visual means to indicate the port speed configuration

(SDR, DDR, QDR, or FDR) and/or the link width (1X or 4X). The speed and link width configurations can be retrieved using management software.

2.3.3

Spine Board LED Indicators

Each leaf board is connected by links to each spine module. Each spine has the following

LEDs.

• One status LED for the spine health

• One status LED for the spine fan modules

• 36 status LEDs showing the existence of the leaf to spine connection

• One Bad Port Led showing symbol errors in the data stream

• One UID LED that can be lit to identify an individual spine

The 36 LEDs on each spine are divided by the number of leafs and the result (N) is the number of connections from each leaf that are connected to the spine. 6 spines must be installed and working to ensure that full BW exists between nodes. The maximum number of connections from each leaf is 3. If the (number of leafs) x (the maximum number of connections per leaf) is less than 36 then some of the leaf to spine connection LEDs may be OFF.


Rev 1.5

25

Rev 1.5

The status LEDs for the spine and their descriptions are shown in Table 8. The LEDs indicate as

follows.

Figure 7: Spine Status LEDs

Spine Status

Hardware

Fan Status

1

3

5 7

9 11

13 15 17

Bad Port

UID

2 4 6 8

10

12

14 16

18

UID

1 3 5 7 9 11 13

15 17

UID

2 4 6 8 10 12 14 16 18

19 21 23 25 27 29 31 33 35

20 22 24 26

28

30

32 34

36

Spine Leaf IB Connection Status

2.3.3.1 Status LED

Table 8 shows the spine status according to the LED condition.

Table 8 - Spine Status LED

LED Condition

Off

Solid Green

Flashing Green

Solid Orange

Solid Red

LED Description

No power to the spine

Spine is up and running

Spine is powering up

Non fatal error – this spine needs troubleshooting, but does not require chassis shutdown

Fatal error

2.3.3.2 Fan LED

The spine fan indicator LED has the following LED assignment:

Table 9 - Spine Fan Status LED

LED Color

Solid Green

Solid Yellow

LED Description

Spine Fan is OK

One or more of the fans in this spine is not working

Each spine has two fans in the fan module



2.3.3.3 Spine to Leaf IB Connection Status LEDs

The leaf connection status on each spine displays the condition of the connection between the spine and each leaf. There is a minimum of one LED per leaf per spine and a maximum of 3 LEDs per leaf. These LEDs indicate a valid connection between a leaf and a spine.

Table 10 shows the leaf to spine status according to the LED condition.

Table 10 - Spine to Leaf IB Link Status

LED Condition

Off

Solid Green

Flashing Green

Solid Orange

Link is down

Logical connection

Data activity

Physical connection

LED Description

2.3.3.4 Bad Port LED

The Bad Port indicator is located on the left side of the spine. The following Bad Port conditions are possible:

Table 11 - Bad Port LED Configurations

LED Condition Description

Off OK – No ports have received symbol errors recently

Flashing Orange Error – One or possibly more ports has just received a symbol error.

This LED shows symbol errors. Possible causes for this are:

• bad cable

• bad connection

• bad connector

This LED lights up when one or more ports is receiving a symbol error. The LED immediately goes off until the next symbol error is received.

2.3.3.5 UID LED Switch Identifier

The UID LED is a debug feature that will become available to customers in the near future. For details please contact Mellanox Technologies support.

2.3.4

Spine Side Panel Display LED Indicators

The spine side panel display has LEDs that show the chassis condition.

Rev 1.5


Rev 1.5

Figure 8: Spine Side Panel Display Status Indications

STATUS

PSU

STATUS

SPINE

FANS

LEAF

FANS

MNG 1

STATUS

MNG 2

STATUS

Table 12 - LEDs Display for Normal Operation


Normal

State

STATUS Off – No Power

Green – System is up and running

Yellow – System warning. Attention needed (such as overheating)

Red – System not operational (Diagnostics fail, CPU hang,



HW fail, Overheat-critical)

Blinking green – System booting / Restore factory defaults in progress

Green

PSU STATUS Off – No power

Green –Normal operational

Red – PS fault detected. User should check individual power supplies for fault indications.

Green

Green SPINE FANS

STATUS

Off – No power to fan

Green – Nominal operational

Red – One or more of the spine fans is bad. User should check

 individual spine fan LEDs for fault indications.

LEAF FANS

STATUS

Green

MNG1



MASTER

STATUS



Red – One or more of the leaf fans is bad. User should check

 individual leaf fan LEDs for fault indications.

Off –

- no power

- this management module is not installed

- this management module is not the master

Green

MNG2



MASTER

STATUS

Green – Management module is operating as a master

Off –

- no power

- this management module is not installed

- this management module is not the master

Green – Management module is operating as a master

Off

Hardware



2.3.5

Management Module LED Indicators

Figure 9: Management Module Status Indications

IB

STATUS

I2C CONSOLE

MGT

RST PSU S.Fans

L.Fans

Master

Status

The management module LEDs display the switch system operating conditions.

Table 13 - LEDs Display for Normal Operation


Normal

State

STATUS

This LED shows the status of the chassis.

Off – No Power

Green – System is up and running.

Yellow – System warning. Attention needed (such as overheating).

Red – System not operational (Diagnostics fail, CPU hang,



HW fail, Overheat-critical)

Blinking green – System booting / Restore factory defaults in progress.

PSU STATUS Off – No power

Green – Normal operational

Red – PS fault detected. User should check individual power supplies

 for fault indications.

Green

Green SPINE FANS

STATUS



Red – One or more of the spine fans is bad. User should check

 individual spine fan LEDs for fault indications

LEAF FANS

STATUS

MASTER



Red – One or more of the leaf fans is bad. User should check

 individual leaf fan LEDs for fault indications

Off – this management module is not the master MM

Green – Management module is operating as a master (i.e., the other

 management module is a slave)

Green

Green

Green

2.3.6

Port Connector Interfaces

The connector side of the switch has 12 leaf boards and each leaf board has 18 QSFP ports. The ports on each leaf board are placed in two rows, 9 ports to a row. The ports are

labelled as shown in Figure 10. The bottom row ports are flipped from the top row. See

Figure 11.

Rev 1.5


Rev 1.5

1

2

3

4

Figure 10: Port Numbering

7

9 1 1 5

1 3

6 8 1 0 1 2

Figure 11: Top and Bottom Ports

1 4

1 5

1 6

1 7

1 8

Hardware

2.4

Air Flow

These switches come with the air flow pattern of air entering through the spine side and exiting through the connector side.

2.5

QSFP Cable Power Budget Classification

All SX6512 QSFP switches are designed for active cables with a max power per module of 2.0W.

This is power level 2 according to the QSFP Public Specification.

2.6

Management Module Interfaces

The switch system requires at least one management module. The management module has five interfaces to connect to the SX6512. They are:

• 1 I2C port

• 1 CONSOLE port – this is an RS232 connector for connecting to a host machine

• 1 MGT – this is an Ethernet connector

• 1 USB port

• 1 RST – reset button

Figure 12: Management Module Interfaces

IB

STATUS

I2C

CONSOLE MGT

RST PSU



2.6.1

I2C

This interface is for Debug and Troubleshooting only.

This interface is for FAEs only.

2.6.2

CONSOLE

The CONSOLE port is used during the installation process to configure the chassis for remote management. Connect this port to a local host using the harness supplied with the chassis. See the Installation Guide for the initial configuration procedure.

2.6.3

MGT– Management

The MGT port is an Ethernet port for remote management. Any remote terminal connected to the Ethernet port can then be used to manage the fabric and chassis.

Each Ethernet connector gets connected to Ethernet switches. These switches must be configured to 100M/1G auto-negotiation.

Rev 1.5

Initial configuration must be done on all of the management modules. The first management module you configure will be the master.

2.6.4

USB

The USB port can be used to upload new SW using any storage device that has a USB connector. This interface is USB 2.0.

2.6.5

Reset – RST

The Reset button resets the chassis management module when the button is pushed.

When the button is held down for 15 seconds the management module is reset and the password deleted.

DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button. Sharp objects can cause damage, use a flat object such as a paper clip.


Rev 1.5

CONSOLE MGT

Figure 13: Reset Button

RST PSU S.FANS

L.FANS

MASTER

Hardware

This button resets the CPU of the management module. A quick push of this button performs this reset. When the reset button is pushed on the master management module this management module is reset becoming the slave and the other management module becomes the master. If there is only one management module in the chassis all of the leafs and ports are reset by bringing them down and powering them up when the reset button is pushed. When the button is held down for

15 seconds the management module is reset and the password is deleted. You will then be able to enter without a password and make a new password for the user “admin”.



3 Installation

This chassis can be installed in standard 19” racks that have depths between 65cm and

80cm between the vertical supports of the rack.

Installation and initialization of the chassis is a simple process requiring attention to the normal mechanical, power, and thermal precautions for rack-mounted equipment. Your chassis comes only with the power supplies and fans pre-installed. The rest of the openings are populated with blanks. All of the leafs, spines, and management modules come shipped in a separate package.

The chassis requires initial configuration to get the chassis and Fabric management up and running through remote management. See the Installation Guide that is packed in the box for the instructions to make the initial configuration.

This unit is intended for installation in a Restricted Access Location. A restricted access area can be accessed only through the use of a special tool, lock and key, or other means of security.

Rev 1.5

3.1

Environmental and Safety Recommendations

The following are Mellanox recommendations.

Recommended ambient temperature in the System room is 20 o

± 5 o

C.

Recommended humidity range is 40% ± 15% without condensing.

It is highly recommended that the installation sites be as isolated as possible from all sources of radio transmissions and electrical interference.

It is highly recommended that the installation site building be equipped with a lightning rod.

It is highly recommended that the installation site be equipped with smoke detectors and a fire alarm warning system.


Rev 1.5

The system requires a KVA rated UPS system. It is recommended that a UPS system be installed to protect the equipment in the event of unexpected power failure.

Make sure that the outlets and circuits will not be overloaded. Spread out the load over at least two or three circuits or use a 3 phase circuit.

3.2

Chassis Package Contents

The package includes:

• 1 box containing

• installation kit parts

• cable management system

• power cords

• 1 shelf

• 1 chassis with the following modules installed:

• 4 fans

• 4 PSUs

If you are not using a mechanical lift to install the chassis, reduce the weight of the chassis by removing all of the power supply units, and fan units, and put aside for reinstallation after the chassis is installed in the rack.

• 1 Installation Guide

• 1 chassis

• 1 box containing various other parts

Inside the single box is 3 smaller boxes.

• 1 installation kit box

• 1 box containing 4 power cords 250v 15a 2.0M, C14 to C13, USA UL Standard

• 1 cable management kit

Before you install your new SX6512 series chassis, unpack the system and check to make sure that all the parts have been sent, check this against the parts list. Check the parts for visible damage that may have occurred during shipping.

If anything is damaged or missing, contact your customer representative immediately.

Installation



The rack mounting holes conform to the EIA-310 standard for 19-inch racks. Guarantee proper ventilation, by leaving 8cm (3”) of space to the front and rear of the switch.

This will ensure proper air flow through the chassis. This is crucial for maintaining good airflow at ambient temperature. In particular, route cables such that they do not impede the air into or out of the chassis.

Rev 1.5

3.3

Leaf Package Contents

The leafs are ordered by the customer and are shipped 4 to a box. The customer will receive as many boxes as needed to fill the order.

3.4

Spine Package Contents

The spines are shipped 3 to a box. The customer will receive as many boxes as needed to fill the chassis with a full complement of spines.

Insert the bottom spine board first.

3.5

Management Package Contents

The package includes:

• all of the management modules ordered by the customer

• 1 RJ45 to DB9 harness for each management module received

3.6

Physical Installation

Warning: This equipment is very heavy. Safety is the first concern. Make sure that adequate manpower and proper equipment is used for transporting and moving the chassis.

The fully loaded chassis weighs:

120 kg (265 LBS) full configuration

48 KG (105 LBS) empty configuration

83.94kg (185 LBS) shipped configuration

The switch platform uses 10U of rack pace in a standard 19” rack, 9U for the chassis and

1U for the shelf. The switch ships from the factory with mounting holes on the spine side. There are upper brackets to connect the leaf side to the rack near the top of the chassis. The weight of the switch is supported from underneath the unit by the shelf.


Rev 1.5

This chassis can be installed in standard 19” racks that have between 65cm and 80cm between the vertical supports of the rack. Make sure that a fully populated rack including cables will have sufficient air flow for cooling.

Installation

Choose a rack which is able to support the mechanical and environmental characteristics of a fully populated switch chassis.

3.6.1

ESD Connection

Before starting any procedure on the SX6512 switch system:

1. Put an ESD prevention wrist strap on your wrist, and make sure there is good contact between your body and the strap.

2. Plug the other end of the wrist strap to a valid ground. Make sure that this is a tight fit.

3.6.2

Installation Procedure

3.6.2.1 Requirements

You will need:

• #2 phillips screwdriver

• a grounding lug

• #3 phillips screwdriver

• ground wire to properly ground the chassis

The installation will be much easier with a power screwdriver.

It is recommended to use AWG6 or 4mm diameter wire for grounding purposes.

It is recommended to have at least two people for the duration of the installation procedure. Use a mechanical lift to raise this chassis. If not, use enough manpower to ensure the safety and wellbeing of all of the people involved in the installation.



3.6.2.2 Installation and Cable Management Kit Parts

Parts for installing the shelf

• 1 shelf

• 12 caged nuts 4 for the shelf and 8 for the faceplate

• 16 M6 bolts for the caged nuts 8 for the shelf and

8 for the faceplate

• 2 Shelf rail slides

Parts for installing the chassis

• 2 chassis rail slides

• 8 M-5 pan head bolts

• 2 flat 4 hole metal spacers

• 8 split lock washers

• 4 handles

• 8 allen head screws

• 1 allen wrench

• 4 lock nuts for handles

Parts for installing the cable manager

• 1 RH cable holder

• 12 caged nuts M6

• 1 LH cable holder

• 12 M6 bolts

You will need 10 U of space in the rack. Nine U for the chassis and one U for the shelf.

Figure 14: Shelf Installation Kit Parts

Rev 1.5

She gled acket

Rail

Rail Slid

M6 X 12 Caged nuts

M6 X 16 bolts


Rev 1.5

Installation

Figure 15: Chassis Installation Parts

L

R

Rail Slide

M-5 Pan head screws

Washer

Screws for the handles

3.6.2.3 Container Mishandling

The container has shock and tilt stickers applied. These will turn red if the container has been mishandled or roughly handled. Upon receipt of the container look for and inspect the shock and tilt stickers to confirm that they have not tripped. If one or more are red notify the shipper and

Mellanox. This on its own does not indicate damage to the contents. But, be sure to carefully inspect the contents if any of the shock and tilt stickers have tripped.



Figure 16: Shock and Tilt Stickers

TIL

T

WA T CH

W

A

R N

I N

G

HANDLE WITH CARE

W

A

R

N I N

HANDLE WITH CARE

Stickers unaffected by

 shipping

Rev 1.5

TIL

T

WA T CH

W

A

R N

I N

G

HANDLE WITH CARE

W

A

R

N I N

HANDLE WITH CARE

Stickers showing excessive shock or tilt.

3.6.2.4 Opening the Container

1. Before starting the procedure, put the ESD strap on and connect it to a valid ground.


Rev 1.5

2. Open the crate by unscrewing the sides and top of the crate.

Container screws

Installation

It is highly recommended to have a screw gun or electric screwdriver for this step.

3. Unscrew the sides of the crate.

4. Remove and put aside the box.

5. Visually inspect the chassis, make sure that:

• there is no visible damage

• 4 PSUs are installed for the chassis

• all 4 fans are installed

6. Remove all protective plastic film from all sides and top of the chassis.

7. If you are not using a mechanical lift to install the chassis, reduce the weight of the chassis by removing all of the power supply units, and fan units, and put aside for reinstallation after the chassis is installed in the rack.

The leafs, spines, and management modules are shipped separately.

3.6.2.5 Installing the Shelf

1. Place the ESD mat on the floor where you will be working and put on the ESD strap. Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground.

2. Clip 4 caged nuts into the holes in the rack you will be using to connect the shelf brackets.

Check that both sides of the shelf are at the same level in the rack.



Figure 17: Inserting the Caged Nuts for the Shelf

21

Top of shelf

20

Bottom of shelf

Do not use this hole

19

3. Using two of the bolts for each rail slide, install the rail slides onto the rack. Check that both sides of the switch, left and right, are the same level in the rack.

Figure 18: Connect Rail Slide to Rack Vertical support

Rev 1.5

4. Place the four bolts for the caged nuts within reach.

5. Put the shelf into place and screw the bolts into the nuts from step 2

6. Extend the rail slides to the rack vertical support and screw them to the rack

7. Tighten all of the screws to 9.2 Nm or 81.5 pound inches.

8. Insert 8 caged nuts for the faceplate in the exact locations shown in

Figure 19,“Inserting the Caged Nuts for the Faceplate”.


Rev 1.5

Figure 19: Inserting the Caged Nuts for the Faceplate

6

5

4

7

3

2

1

9

8

Place the caged nuts here.

Shelf

3.6.3

Installing the Cable Holder

Now is the time to install the cable holder.

Installation



Figure 20: Cable Holders

Right side cable holder

12 X M6 caged nut

12 X M6 bolt for caged nut

Rev 1.5

Left side cable holder

1. Place the Cable holder next to the rack, on the connector side of the chassis, and identify the holes where the caged nuts are to be placed.

The cable holders should go to the outside of the vertical supports.

2. Set the bottom of the cable holder at the level of the shelf.

If the cable holder is not set properly the upper bracket will not line up with the cable holder.


Rev 1.5

Figure 21: Cable Holder Placement

Notch for upper bracket washer

L

Set this even with the top of the shelf

3. Screw the cable holder onto the rack using the screws provided.

4. Repeat steps 1- 3 for the second cable holder.

Installation



3.6.4

Installing the Chassis

Figure 22: Chassis Rails and Rail Slides

L

R

Rail Slide

Rev 1.5

Do not use the handles on the fans for lifting or moving the chassis in the rack!


Rev 1.5

Figure 23: Placement of Chassis in Rack

Doors

RACK

19" (482.6mm)

Shelf

Connector side

Cable with recommended bending radius

Chassis

Spine side

Insufficient room for recommended bending radius

Doors

You will most likely need extra room for cable bending on the connector side of the chassis and should plan to keep the spine side as close to the rack door as possible, thus having more room

for the cables. See Figure 1 on page 46.

1. Screw the handles onto the chassis. Use the 8 allen head screws, and the allen wrench provided. Two handles go on the connector side and two go on the sides of the chassis. Use the locknuts on the spine side of the chassis to secure the handles. There may be extra nuts left over.

Figure 24: Installing the Handles

Installation

There may be extra nuts left over. Don’t worry.



Figure 25: Screw the Handles Onto the Chassis

Handles

L

Rev 1.5


Mellanox

®

L

PS1

PS3

Fan #1

S1

S2

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1

3

5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

Fan #2 Fan #3

19 21 23 25 27 29 31 33 35

20 22 24 26 28 30 32 34 36

19 21 23 25 27 29 31 33

35

20 22 24 26 28 30 32 34 36

19 21 23 25 27 29 31 33 35

20 22

24

26 28 30 32 34 36

19 21 23 25 27 29 31 33 35

20 22 24 26 28 30 32 34 36

19 21 23 25 27 29 31 33 35

20 22 24 26 28 30 32 34 36

19 21 23 25 27 29 31

33 35

20 22 24 26 28 30 32 34 36

Fan #4

SX6512

R

PS2

PS4


Rev 1.5

Figure 26: The Rails are Already Connected Onto the Chassis

R

Installation

When lifting manually use your legs, bend your knees, and not your back.

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

70 - 121 lbs

32 - 55 kgs

>121 lbs

>55 kgs

2. Lift the chassis and slide it onto the shelf. Use a mechanical lift or enough people to safely lift the chassis. A full chassis weighs ~120 kgs an empty chassis weighs ~50 kgs.




Figure 27: Raising the Chassis Using a Mechanical Lift

Rev 1.5

3. Remove the two handles from the sides of the chassis.

4. Push the chassis into the rack until the faceplate is ~ 20cm (~ 8”) from the vertical support.

5. Put the rail slides onto the rails.

Figure 28: Put on the Rail Slide

R

R


Rev 1.5

6. Slide the chassis further into the rack.

7. Slide the chassis all the way into the rack until the faceplate is touching the vertical rack support. The caged nuts placed in the last procedure should line up with the holes in the faceplate.

8. Screw the 8 screws through the faceplate and into the caged nuts.

9. Slide the rail slides to the vertical rails.

Figure 29: SX6512 Chassis on the Shelf

Installation

R

50

Figure 30: Face Plate Mounting Bolt Locations

Holes in the faceplate for mounting on the rack.

Mellanox

®

L

PS1

PS3

Fan #1

S1

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3

5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18


SX6512

R

PS2

PS4



Figure 31: Move the Rail Slide to the Vertical Support

Rev 1.5

R

10. Place the 4 holed spacer on the outside of the vertical support and screw in the 4 screws for each rail slide. Use the split lock washers here.

11. Remove the two handles from the spine side and save all four handles for future use.

12. Ground the switch.

13. Replace all of the spines, power supply units, fan units, and management modules removed at the start of the installation procedure.

14. Plug in the power cables.

15. Check the Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation.

Warning: Any yellow status LEDs is cause for concern and must be dealt with immediately.

It can take up to 5 minutes to boot up, during which time the status LED may indicate flashing green.

16. You can start connecting all of the cables to the switch.

3.6.5

Ground Connections

Make sure to connect the ground post to a valid electrical ground. Use a grounding lug and a ground wire of sufficient capacity to safely convey a potential discharge. A ground wire of AWG 6 or 4mm diameter is recommended for grounding this device. The chassis is concurrently grounded through each of the PSUs. Only connect the PSU cords to properly grounded outlets. Do not rely on the PSU grounds. It is absolutely necessary to connect the grounding post. Make sure the connections are solid and permanent. If you choose to not use the ground screw, make sure that the rack is properly grounded and that


Rev 1.5

there is a valid ground connection between the chassis of the switch and the rack. Test the ground using an Ohm meter.

Figure 32: Ground Connection

Installation

Mellanox

®

Fan #1

S1

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3

5 7

9 11 13

15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

Fan #2

ESD connector

Fan #3 Fan #4

SX6512

R

Ground Post

3.7

Power Connections

The switch includes integrated hot-swap power supplies which support up to 4 load-sharing

1000W or 1600W supply units. The slots for the power supply units (PSUs) are on the spine side.

The left side has odd numbered PSUs and the right side has even numbered PSUs. Each PSU has a dedicated AC inlet. This design enables the optional use of separate main and backup AC feeds.

The input voltage is Autorange, 100-240 VAC, 50Hz or 60Hz. The output voltage for the PSUs is

48V. The power cords should be standard 3-wire AC power cords including a safety ground.

3.7.1

Powering Up the Switch Platform

Make sure that the power cords are compatible with your outlets. Power cords for different countries can be ordered from Mellanox.

The chassis in N+1 mode must be started with a full complement of PSUs, thereafter it can run on one less than the total number of PSUs. This final PSU is redundant and allows for hot swapping a PSU should one fail. Connecting the PSUs to different AC lines provides AC failover protection.

When using the 1600W PSUs, the minimum complement of power supply units to start the chassis is 2 and the chassis will continue to run at full capacity with only 2 PSUs.

The system should continue to run and allow a hot swap of a defective PSU. Should there not be enough power to keep all of the leafs running, MLNX-OS™ may power down some leafs. If this happens it may be necessary to reboot the chassis once the defective PSU has been replaced. Two simple ways to reboot is to use the reboot command in the CLI or reboot through the Web GUI.



The power system will divide the current consumption by the number of working PSUs.

Should one of the PSUs fail, the total current consumption will then be divided by the remaining working PSUs. When the failed PSU is hot swapped the new PSU will ramp up and pass its share of current, so that the total current is always divided by the number of working PSUs.

1. Plug in the power cords to the PSUs.

2. Plug the other end of the power cords into grounded outlets.

Make sure that the outlets and circuits will not be overloaded. Spread out the load over at least two or three circuits or use a 3 phase circuit.

Figure 33: Multiple Power Inlets - Electric Caution Notification

CAUTION




ACHTUNG

Gafahr des elektrischen Schocks.

Entferrnen des Netzsteckers elnes Netzteils spannungsfrei.

Um alle Einhieten spannungsfrei zu machen sind die Netzstecker aller Netzteile zu entfernen

ATTENTION

Risque de choc et de danger

 e’lectriques. Le de’branchment d’une seule alimentation stabilise’e ne de’branch uniquement qu’un module “Alimentation

Stabilise’e”. Pour isoler completement le module en cause, Il faut de’brancher toutes les alimentations stabilise’es.

3. Check the Status LEDs on all of the management modules and confirm that all of the

LEDs show status lights consistent with normal operation.

Rev 1.5

Any yellow or red status LEDs on any of the management modules is cause for concern and must be dealt with immediately.

4. Check that none of the LEDs on the spines are yellow.

It can take up to 5 minutes to boot up the system. Turn off the system if any LEDs remain red for more than 5 minutes.

5. Check that the Leaf status LEDs, Fan status LED, and Spine status LED in the spines are all green.

The maximum number of connections from each leaf is 3If the (number of leafs x the maximum number of connections per leaf) is less than 36 then some of the leaf to spine connection LEDs may be OFF.


Rev 1.5

Figure 34: Spine Module

1 3 5 7 9 11 13

15 17

UID

2 4 6 8 10 12 14 16 18

19 21 23 25 27 29 31 33 35

20 22 24 26

28

30

32 34

36

Status, Fan, Bad Port and UID LEDs

Leaf connection LEDs

3.8

InfiniBand QSFP Cable Installation

The switch uses industry standard QSFP InfiniBand cables which are available from Mellanox

Technologies. The Mellanox proprietary QSFP cables support full 56+56Gb/s (FDR), 40+40Gb/ s (FDR10), 40+40Gb/s (QDR), 20+20Gb/s (DDR) and 10+10Gb/s (SDR) bidirectional wire speed of the switch ports. All InfiniBand QSFP connections are made to the leaf boards. Each

leaf has 18 InfiniBand QSFP connectors in two rows, which are numbered 1-18. See Section

2.3.6 for port numbering.

Installation

If maximum cable lengths are exceeded data transfer will be reduced and the bit error rate will increase.

FDR BandWidth is supported only by MLNX QSFP cables.

54

FDR and FDR10 are only guaranteed to work with approved Mellanox Cables.

All cables can be inserted or removed with the unit powered on. To insert a cable, press the connector onto the port receptacle until the connector is firmly seated. The orange LED indicator above the port will light when the physical connection is established (when both ends of the cable are properly connected to working devices). Allow 15 seconds for link to get up. To remove, disengage the lock and slowly pull the connector away from the port receptacle.

For a valid physical connection both ends of the cable must be connected to working devices.

Take care to not impede the air flow through the ventilation holes next to the Infini-

Band ports. Use cable lengths which allow routing horizontally around to the side of the chassis before bending upward or downward in the rack.



3.8.1

Supported Approved Cables

For a list of approved cables for this switch see the Mellanox approved cable list.

http://www.mellanox.com/related-docs/user_manuals/Mellanox_approved_cables.pdf

3.8.2

Cable Power Classes

Chassis and switches need to be able to dissipate the heat generated by high power I/O cables and modules. The Mellanox SX 65XX series chassis are rated for cables up to class 2 as per the SFF committee classification ( SFF-8436.PDF

).



See http://www.mellanox.com/content/pages.php?pg=cables for the cable class rating of

Mellanox cables.

Rev 1.5


Chassis Power Up Rev 1.5

4 Chassis Power Up

Before starting any procedure on the SX

6512

system put an ESD prevention wrist strap on your wrist and connect to the SX

6512

chassis.

With N+1 PSU redundancy the chassis must be started with a full complement of possible PSUs, thereafter it can run on one less than the total number of PSUs. This final PSU is redundant and allows for hot swapping a PSU should one fail. Connecting the PSUs to different AC lines provides AC failover protection.

The system should continue to run and allow a hot swap of a defective PSU. Should there not be enough power to keep all of the leafs running, MLNX-OS™ may power down some leafs. If this happens it will be necessary to reboot the chassis once the defective PSU has been replaced.

With 1600W optional PSU supplies the chassis can run on as little as 1/2 of the full complement of PSUs. N+N allows the chassis to run on 2PSUs supplied from one power grid while 2 are connected to a second power grid.

1. Check all FRUs for proper insertion and seating before connecting the AC power cords.

• Boards

• Power supplies

• Leaf fan modules

• Spine fan modules

2. Insert all leafs that you plan to use, in the chassis. Start at the bottom of the chassis and work your way up.

3. Insert the first two spines in the top two slots.

4. Insert the rest of the spines from the bottom of the chassis up to slot #3.

5. Insert thermal blanks in unused leaf slots to maintain balanced air flow.

6. Tighten all leaf and spine mounting screws.

7. Connect the power cords to the PSUs.

8. Connect the power cords to grounded electrical outlets.

With N+1 PSU redundancy do not power up the chassis with less than all PSUs installed.

4.1

Power Supply and Spine Board Indicator Status at Power ON

It can take up to 5 minutes to boot up the system. Turn off the system if any LEDs remain red for more than 5 minutes.



As the power is turned on, you should observe the following conditions for normal operation:

1. Power Supply Unit(s) AC OK and DC OK indicators are ON and FAIL indicators are

OFF.

2. There is a green Status LED per spine board, per leaf board, and per management module that indicates power supplies are good.

3. Spine Board indicators will display status of internal links to the installed leaf boards.

All PHY links to existing leaf Boards should be ON.

4. Check the Spine LEDs and make sure they coincide with Figure 35.

Figure 35: Spine Side Panel Display Status Indications

Rev 1.5

STATUS

PSU

STATUS

SPINE

FANS

LEAF

FANS

MNG 1

STATUS

MNG 2

STATUS

5. Check the Management Module LEDs and make sure they coincide with Figure 36.

Figure 36: Management Module Status Indications for Normal Operation

IB

STATUS

I2C CONSOLE MGT

RST PSU S.Fans

L.Fans

Master


Rev 1.5

Insertion and/or Hot swap Extraction

5 Insertion and/or Hot swap Extraction

Before starting any procedure on the SX6512 switch system, put an ESD prevention wrist strap on your wrist and connect to the SX6512 chassis.

Do NOT mix replacement parts based on different generations of chip.

Do not install InfiniScale IV based replacement parts within a SwitchX based chassis and vice versa. All Replacement modules must be consistent with the Chassis family and switch chip generation.

When hot-swapping any of the units, it is necessary to wait 1 minute after removing the defective part before inserting the new part. This is necessary so that the management module will start a new cycle checking through the leafs and spines for the FW versions.

This switch platform supports hot swap capabilities for the parts listed below.

• Power supply units

• Leaf boards

• Spine boards

• Leaf fan module

• Spine fan module


5.1

Power Supply Units

The 1000 W power supplies deliver 1000W at 48V. The input to each of these power supplies requires 1214.46W at 220V in order to output 1000W.

For N+1 configuration PSUs (Power Supply Unit) can be extracted without bringing down the system.

For N+N configuration up to half of the PSUs can go down and the system will continue to run.

The power required to run the switch system is equally divided between all of the working PSUs.

5.1.1

Extracting and Inserting the Power Supply Unit

With all of the 1000W power supplies installed, the system is in N +1 redundant configuration.

With all of the N+N 1600W power supplies installed, the system is in N +N redundant configuration, half of the PSUs may be extracted without bringing down the system.To extract a PSU:

1. Determine which AC connector on the connector side of the chassis corresponds to the defective PSU.

2. Remove the power cord from the power supply unit. Note which power cord it is according to the AC numbering.



Figure 37: Power Cord Numbering

AC1

AC2 AC3 AC4

Figure 38: PSU Locations

PSU 1

PSU 3

Mellanox

®

Fan #1

S1

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

S2

S3

S4

S5

S6

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID 2 4 6 8 10 12 14 16 18

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18


SX6512

R

PSU 2

PSU 4

3. On the spine side of the chassis, remove the cover to the power supply unit for the non-working PSU. Odd numbered PSUs are on the left side and even numbered PSUs are on the right side facing the spines.There are four phillips head screws for the cover plate.

Figure 39: Power Supply

Handle

OK

DC

FAIL

AC

OK

Latch

Rev 1.5

4. Grasping the handle with one hand, push the black latch release while pulling the handle outward. As the PSU unseats, the PSU status indicators will turn off.


Rev 1.5


5. Remove the PSU.

To insert a PSU:

1. Make sure the mating connector of the new unit is free of any dirt and/or obstacles.

2. Insert the PSU by sliding it into the opening until a slight resistance is felt.

3. Continue pressing the PSU until it seats completely. The latch will snap into place confirming the proper installation.

4. Insert the power cord into the supply connector on the other side of the chassis.

5. Replace the cover over the PSUs.

The green indicators should light. If not, extract the PSU and re-insert it again.

5.2

Leaf Boards




The leaf boards are numbered from top to bottom, with corresponding numbers displayed to the outside of the leafs vertically along the side panel.

L1

L3

F

D

R

Figure 40: Leaf Board Numbering

F

D

R

1

2

3

4

5

6

7

8

9

10 11 12

13

14 15 16 17 18

1

2

3

4

5

6

7

8

9

10 11 12

13

14 15 16 17 18

UID

F

D

R

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

UID

UID

F

D

R

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

UID

L2

L4

L11

F

D

R

UID

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

F

D

R

UID

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

L12

5.2.1

Extracting a Leaf Board

Each leaf board has an ejector that locks the board in place and serves as a lever for seating or

extracting (see Figure 51).

1. Run the shut down command

“no power enable <module>”.

For example to shut down leaf 16 run the command below.

switch-1 [Orca648: master] (config) # no power enable L16



2. Disconnect all cables connected to the leaf.

3. Push the latch to unlock the ejector from the chassis.

4. Open the ejector until it is 45 degrees from the leaf.

5. Pull out on the leaf.

Figure 41: Ejector Latch

Pull the leaf out of the chassis

Leaf

Leaf

Push the latch release

Ejector

The board is short, therefore do not let go of it while sliding it out.

Push here to release the latch

Figure 42: Leaf Release

Pull the ejector to remove the leaf

6

7

8

9

10

11

12

13

14

15

16

17

18

Rev 1.5

5.2.2

Inserting a Leaf Board

Start with the latch fully open, that is 45 degrees to the front panel of the leaf. Carefully set the leaf board into the side guide rails and slowly slide the board into the chassis.

Catch the hooks onto the vertical bar and push the latch shut.


Rev 1.5

5.3

Spine Boards




When hot swapping any of the units, it is necessary to wait 1 minute after removing the defective part before inserting the new part. This is necessary so that the management module will start a new cycle checking through the leafs and spines for the FW versions.

Each spine has a pair of ejectors that lock the board in place and serve as levers for seating or

extracting (see Figure 44).

Management board #1 is connected to spine board #1, and management board #2 is connected to spine board #2. When a slave management board is not installed or not working, the spine

board connected to the master management board cannot be hot-swapped. All of the spine boards can be hot-swapped when two management boards installed and working.

When more than one spine slot is empty always insert the lowest spine board first, then work your way up.

5.3.1

Extracting a Spine Board

When a slave management board is not installed or not working, hot-swapping the spine board connected to the master management board will cause the chassis to crash.

Neither the CLI nor the GUI management tools will allow you to shut down spine #1 or spine #2, as the management modules are connected to the chassis components through these spines.

5.3.1.1 Extracting Spine Board #1 or #2

Spine board #1 is connected to management board #1 and spine board #2 is connected to management board #2.



Figure 43: Management Module Numbering

D

D

IB

STATUS

IB

STATUS

I2C CONSOLE

I2C CONSOLE

MGT

MGT

RST PSU S.Fans

L.Fans

Master

RST PSU S.Fans

L.Fans

Master

MGT MAC:

1

2

3

4

5

6

7

8

9

10 11 12

13

14

15

16 17 18

1

2

3

4

5

6

7

8

9

10 11 12

13

14

15

16 17 18

1

2

3

4

5

6

7

8

9

10 11 12

13

14

15

16 17 18

MGT MAC:

F

D

R

1

2

3

4

5

6

7

8

9

10 11 12

13

14

15

16 17 18

F

D

R

UID

1

2

3

4

5

6

7

8

9

10 11 12

13

14

15

16 17 18

F

D

R

UID

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

MNG1 management module #1

MNG2 management module #2



(bottom)

Rev 1.5

Removing spine #1 causes management module #1 to reset.

Removing spine #2 causes management module #2 to reset.


Warning: If the spine you want to hot swap is connected to the master management module the management module will reboot when you take out the spine.



If you have only one management module the chassis will crash!


Pull the module out of the chassis

Management Module /Spine


Figure 45: Spine Board Extraction

Pull the latches to remove the spine.

19 21 23 25 27 29 31 33

35

20 22 24 26 28 30 32 34 36

1 3 5 7 9 11 13 15 17

UID

2 4 6 8 10 12 14 16 18

If you need to hot swap spine #1 or spine #2:


Rev 1.5


1. Check to see if the spine you need to remove is connected to the master management module.

See Figure 43.

Warning: If you have only one management module, the chassis will crash!

2. Push outward on the latches to unlock the ejectors from the chassis.

3. Open the ejectors until they are 45 degrees from the module.

4. Pull out on the module using both ejectors.

5.3.1.2 Extracting Spine Boards Except #1 or #2



For example to shut down spine 06 run the command below.

switch-1 [Orca648: master] (config) # no power enable S06




5.3.2

Inserting a Spine Board

1. Start with the latches fully open, that is 45 degrees to the front panel of the leaf.

2. Hold the spine board by the sides with both latches fully open.

3. Insert the board into its rail. The board should slide in without resistance.

4. Push the board in until the hooks on the latches are touching the vertical bar.

5. Now press both latches inward simultaneously. You should feel the latches are hooked into the front panel.

6. Press both latches applying some force to close them completely. The locking latch will engage.

5.4

Fan Modules

There are four fan modules on the chassis for the leafs. They are located on the spine side, along the top of the chassis. When a fan module is not functioning the status LED on the fan will light up.

Each spine has an individual fan module that contains two individual fans. Should a single fan fail the Fan Status LED on the spine and the S. Fan LED on the management module will light, indicating the necessity to replace the fan module. Air flow through the spines is independent of the air flow through the leafs.



Figure 46: Fan Locations on the Chassis

Fans

Mellanox

®

5.4.1

Leaf Fan Module

There are 4 Leaf fan modules found on the Spine side.

5.4.1.1 Extracting the Leaf Fan Module

1. Push and hold the blue latch release. See Figure 47.

2. Slowly pull out the fan module using the handle.

Figure 47: Leaf Fan Module Extraction

Handle

Latch

Rev 1.5

5.4.1.2 Inserting the Leaf Fan Module

1. Make sure the fan module is oriented correctly top side up. Confirm that the location of the connector in the chassis will line up with the connector in the fan module.


Rev 1.5


2. Slowly slide in the new leaf fan module.

If the fan module stops before it goes in all of the way it is inserted incorrectly or it may be for the other side of the chassis!

3. Push the fan module until the latch engages.

4. Make sure that the green leaf LED on the module comes on (indicating that fan is running).

5.4.2

Spine Fan Modules

Each spine module has a fan module with two individual fans built in. When a fan module is not functioning the Fan Status LED on the spine will light up.

Figure 48: Spine Fan Module

1 3

5 7

9 11

13 15 17

UID

2 4 6 8 10 12 14 16 18

19 21 23 25 27 29 31 33 35

20 22 24 26 28 30 32 34 36

Fan Module

When a fan module is removed the indicator light will reset.

Figure 49: Fan Status LED on the Spine Module

1 3

5 7 9 11 13

15 17

UID

2 4 6 8

10

12

14 16

18

Fan Status

5.4.2.1 Extracting the Spine Fan Module

1. Push the two blue latch buttons together while pulling the fan module out.

5.4.2.2 Inserting the Spine Fan Module

1. Make sure the fan module is oriented correctly top side up. Confirm that the location of the connector in the chassis will line up with the connector in the fan module.

2. Slowly slide in the new spine fan module.



3. Push the fan module as far as it will go, make sure the locking latches engage.

If the Fan LED continues to show red remove the fan module and check the pins on the connector inside if the spine to make sure that none of them are bent.

Rev 1.5

5.5

Management Module


Before starting any procedure on the SX6512 series switch system put an ESD prevention wrist strap on your wrist and connect to the SX6512 series chassis.

5.5.1

Extracting a Management Module

Management modules are located on the leaf side, above the leafs. There are two places to install the management modules.

Only one management module is required to run the switch system.

Each management module has a pair of ejectors that lock the board in place and serve as

a lever for seating or extracting (see Figure 51).



For example to shut down management module 2 run the command below.

switch-1 [Orca648: master] (config) # no power enable MGMT2

2. Disconnect all cables connected to the management module.





Rev 1.5




Pull the module out of the chassis



The board is short, therefore do not let go of it while sliding it out.

Figure 51: Management Module

IB

STATUS

I2C CONSOLE MGT

RST PSU S.Fans

L.Fans

Master

Ejectors

5.5.2

Inserting a Management Module

1. Start with the latch fully open, that is 45 degrees to the front panel of the leaf.

2. Carefully seat the management module into the slot and slowly slide the board into the chassis until the ejectors begin to engage on the chassis edge.

3. Press the ejectors inward until the locks snap. This indicates that the board is fully seated.

4. Perform the initial configuration on the management module. See the Installation Guide.

On switch systems with dual management systems, first connect the cable and configure the master management module CPU and only then configure the slave. By default the master CPU is the top management module. For further information on the master and slave roles, see the

MLNX-OS™

Software UM section “High Availability”.



All management modules in the chassis must go through an initial configuration procedure. See the Installation Guide for the initial configuration procedure.

5.6

Switch Shut Down Procedures

To shut down the chassis run the following command twice (once for each MM):

Reload halt [noconfirm]

The chassis cannot be restarted remotely!

To restart the chassis you must physically go to the switch and unplug all of the power cords to the chassis and then replug in all of the power cords to the chassis.

The first time you run the command it shuts down the master management module and the second time shuts down the slave management module.

To shut down a leaf run the following command:

no power enable <module>

To shut down a spine run the following command:


To shut down a management module run the following command:


Rev 1.5


Disassembly and Disposal Rev 1.5

6 Disassembly and Disposal

6.1

Disassembling the Chassis

1. Power down the chassis.

2. Remove all power cables.

3. Remove all connector cables.

4. Disconnect the ground lug from the ground post.

5. Remove all leafs.

6. Remove all spines.

7. Remove all management modules.

Use enough people to safely lift this product.

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

70 - 121 lbs

32 - 55 kgs

>121 lbs

>55 kgs

This product and all of its parts are NOT to be disposed of with household waste. This product contains printed circuit boards cables and batteries. According to the WEEE

Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and disposed of according to the directive.

8. Go to the Mellanox website for detailed instructions for disassembly of the FRUs and chassis according to the WEEE Directive.

9. Dispose of these pieces in a legal and environmentally friendly way.

6.1.1

Removing the Chassis

1. Remove the screws connecting the upper brackets to the rack.

2. Remove the screws holding the chassis to the rack. These screws are located in the faceplate on the spine side of the chassis.

3. Remove the upper brackets from the chassis.

4. Dispose of the Chassis in a legal and environmental way.

5. Go to the Mellanox website for detailed instructions for disassembly of the chassis according to the WEEE Directive.

6.1.2

Removing the Bottom Shelf

1. Remove all of the bolts that are holding the shelf to the rack.

2. Remove the shelf.

3. Remove all of the caged nuts.



6.2

Disposal

According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not disposed of with regular household waste.

Dispose of this product and all of its parts in a responsible and environmentally friendly way.

Go to the Mellanox website for detailed instructions for disassembly according to the

WEEE Directive.

Rev 1.5


Rev 1.5

Switch Management Tools

7 Switch Management Tools

This chapter describes the management module and tools available for Out-of-Band management of the switch system via MLNX-OS.

There are 2 Ethernet ports (1 for each management module) that get connected to

Ethernet switches. These switches must be configured to 10/100M auto-negotiation.

The SX6512 switch comes standard with a management software module for chassis management called Mellanox Operating System (MLNX-OS). MLNX-OS is installed on all SwitchX based managed switch systems management modules. MLNX-OS includes a CLI, WebUI,

SNMP, and chassis management features for software and IB management software (OpenSM).

You can get more information via the Mellanox MLNX-OS™ SwitchX® Software WebUI

User’s Manual or the Mellanox MLNX-OS™ SwitchX® Software User Manual.

The managed switch system includes the following software components:

• Embedded Subnet Manager (SM)

• Chassis manager and system BIST

• SNMP agent, 3rd party tool integration

• GUI

• Remote logging

• SSH/telnet

• Secured access in-band and out-band

• IPv4/IPv6 network stack

The chassis manager will give the user access to:

• Switch temperatures

• Power supply voltages

• Fan unit information

• Power unit information

• Flash memory

• Monitoring of:

• AC power to the PSUs

• DC power out from the PSUs

• chassis failures

• querying for:

• switch serial numbers

• revisions

• software version

• SwitchX FW version

• switch temperatures

The manager also has the ability to burn new firmware and upgrade software on the switch.



7.1

InfiniBand Subnet Manager

The InfiniBand Subnet Manager (SM) is a centralized entity running in the switch. It discovers and configures all the InfiniBand fabric devices to enable traffic flow between those devices. The SM applies network traffic related configurations such as QoS, routing, partitioning to the fabric devices.

You can view and configure the Subnet Parameters (SM) via the CLI/WebUI.

Each InfiniBand subnet needs one subnet manager to discover, activate and manage the subnet. An InfiniBand® network requires a Subnet Manager to be running in either the

Infiniband switch itself (switch based) or on one of the nodes which is connected to the

Infiniband fabric (host based).

7.2

Fabric Inspector (Diagnostics)

Fabric Inspector is a plug & play software for fee within MLNX-OS displaying and filtering all identified systems and nodes within the fabric.

Fabric Inspector includes a complete set of tools for fabric wide diagnostics to check node-node and node-switch connectivity and to verify routes within the fabric.

Advanced filtering allows creating filtering rules on a system wide basis, between nodes or port connections based on traffic patterns and user assigned system names (GUIDs).

7.3

Accessing the CPU via the Ethernet Connector

Once the initial configuration is completed the management tools can be accessed through:

• SSH

• Telnet

• WEB

• SNMP

• XML

7.4

Upgrading Software

The new software and firmware update is available to the user from the Mellanox Support website. Copy the update to a known location on a Remote server within your LAN.

Use the CLI or the GUI in order to perform the Software upgrade. For further information please refer to the Mellanox-OS user manual.

If

Mellanox-OS is updated and the FW image in the leafs and spines of the chassis is an earlier version than the minimum that the new version of the software can work with, then the chassis management system may require up to ~45 minutes to update all of the

FW images in all of the leafs and spines.

Rev 1.5


TroubleShooting Rev 1.5

8 TroubleShooting

8.1

Power Supply Unit

If the Power supplies cannot supply enough power the management module may shut down the system.

As each PSU is plugged in, make sure that the green power LEDs on the PSU comes on.

If the AC power LED is off:

1. Check that the power cable is the correct power cable for your country.

2. Check that the power cable is plugged into a working outlet.

3. Check that the power cable has a voltage within the range of 100 - 240 volts AC.

4. Remove and reinstall the power cable.

5. Check the circuit breakers to be sure that the breaker has not tripped.

6. Check that the power cable is good. Replace the power cable.

7. If the AC power LED is green but the OK power LED is off or the FAIL LED is on – Replace the PSU.

8.2

Leaf Board

The power LED for the Leaf board is off:

1. Make sure that all of the PSUs are showing DC OK.

2. Uninstall and reinstall the Leaf board.

3. When the Yellow LED is on, this indicates a fault in the board, uninstall and reinstall the Leaf board.

4. If uninstalling and reinstalling the Leaf board does not work, burn the latest FW on the Leaf board and uninstall and reinstall the Leaf board.

5. Replace the Leaf board with a new one.

Should any of the boards shut down due to over temperature, wait 5 minutes and then

follow the procedure starting with step 2

74

The Physical link LED for the InfiniBand connector does not come on:

1. Check that both ends of the cable are connected.

2. Check that the locks on the ends are secured.

3. Make sure that the latest FW version is installed on both the HCA card and the switch.

4. If media adapters are used check that the all connections are good, tight, and secure.

5. Replace the cable.

The Activity indication does not come on:



Check that the Subnet Manager has been started.

8.3

Management Module

8.3.1

Yellow Status LED for the Chassis on the Management Module is Lit

1. Check the MLNX-OS management for confirmation and possible explanation of the alert.

2. Reset the master management module by pushing the rest button. If you have two management modules installed this will convert the master management module to the slave and convert the slave to the master.

If there is only one management module in the chassis all of the leafs and ports are reset by bringing them down and powering them up when the management module is removed.

3. Make sure the S.Fans and L.Fans LEDs are green.

4. Make sure that the spine and the leafs both have the same version of FW.

5. Reburn the FW and remove and reinstall the management module.

6. If you are running the chassis with only one management module, remove and reinstall the management module. Make sure the mating connectors of the unit are free of

any dirt and/or obstacles. See Section 5.5 on page 67.

7. If you are running the chassis with only one management module, replace the management module.

8.3.2

Yellow LED for the Leaf Fan on the Management Module is Lit


2. Make sure that there is nothing blocking the front or rear of the chassis and that the fan modules and ventilation holes are not blocked (especially dust over the holes).

3. If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the front and rear panels of the switch using a vacuum cleaner.

4. Determine which fan module is problematic by checking the status LED on each fan module.

5. Remove and reinstall the problematic fan unit. Make sure the mating connector of the

new unit is free of any dirt and/or obstacles. See Section 5.4 on page 64.

6. Replace the Leaf fan module.

Rev 1.5

Replace defective leaf fan modules as soon as they are identified.

Should any of the boards shut down due to over temperature, follow the procedure

starting in step 2


Rev 1.5

TroubleShooting

8.3.3

Yellow LED for the Spine Fan on the Management Module is Lit


2. Determine which spine has a defective fan by checking the Fan LEDs on all of the spines.



5. Remove and reinstall the fan unit of the spine. Make sure the mating connector of the new

unit is free of any dirt and/or obstacles. See Section 5.4 on page 64.

6. Replace the spine fan module.

Replace defective spine fan modules as soon as they are identified.

8.4

Spine Board

The yellow LED on the Spine board is lit:




4. Remove and reinstall the spine board. Make sure the mating connectors of the unit is free of

any dirt and/or obstacles. See Section 5.3 on page 62.

5. Make sure that the spine and the Leafs both have the same version of FW.

6. Reburn the FW and remove and reinstall the spine.

7. Replace the spine board.

8.5

MLNX-OS™ Software

For more detailed instructions concerning MLNX-OS™ software see the Mellanox MLNX-

OS™ SwitchX® Software WebUI User’s Manual or the Mellanox MLNX-OS™ SwitchX®

Software User Manual.



Appendix A: Specification Data

Specification Data Sheet

Table 14 - Switch Specification Sheet SX6512-[4N]R

Physical

Size / Mounting

Weight /

Center of

Gravity

Air Flow

Size:

Weight:

Sound Level /

Ports

Sound Level

15.75” x 19” x 27” inches

400 mm x 482.6mm x 733 mm

The shelf adds 44.5mm to the height

Mounting:

19” Rack mount

120 kg (265 LBS) full configuration

48 kg (105 LBS) empty configuration

83.94 kg (185 LBS) shipped configuration

Center of Gravity: CoGh

311.8 (mm from bottom of chassis) Measured from the bottom left corner as one faces the switch ports.

CoGw

222 (mm from left side surface)

CoGd

328.14(mm from port end surface)

Measured from the bottom left corner as one faces the switch ports.

Max. Air Flow

Through leafs: 16.59M3/min. (585.9 CFM)

SerDes Speeds /

Connector Types

SerDes Speeds:

10, 20, 40,Gb/s or 56,Gb/s per port

76 db(A)

Max. Air Flow

Through spines: 9.48 M3/min. (334.8 CFM)

Connector

Types: QSFP

Ports

216 ports

Input Voltage /

Management

Input Voltage:

Temperature

Operating

Cable power /

Humidity

QSFP:

Power and Environmental

Management

CPU: 100 - 240 VAC 50-60Hz

0° to 45° Celsius

MAX 2W

TYP: 1.5W

PowerPC 460EX

Non-operating -40° to 70° Celsius

Humidity:



Operating

5% - 95% non-condensing

Rev 1.5


Rev 1.5

Total Power

Consumption:

Typical FDR14 Passive cables:



2287W

Typical FDR14 Active optical cables:



2846W (including QSFP at 1.5W)

Power Consumption Leaf: Typical:91W



(not including QSFP power)

Max: 94W



(not including QSFP power)

Power numbers

Max FDR14 Passive cables:



2390W

Max FDR14 Active optical cables:



3047W (including QSFP at 2.0W)

Power Consumption Spine: Typical: 108W (include 2 Fans)

Max Heat Output: 10,397 BTUs/hr

Max:117W (include 2 Fans) Power Consumption for

Active cables

QSFP:

2.0W MAX Power level #2

Protocol Support

InfiniBand: Management:

Speed protocol /

Management

Data Rate / QoS

Data Rate:

Auto-Negotiation of (56Gb/ s,40Gb/s, 20Gb/s, 10Gb/s)

Managed using MLNX-OS

Baseboard, Performance, and

Device management Agents for full InfiniBand In-Band Management

8 InfiniBand Virtual Lanes for all ports

Safety / EMC

(Emissions)

Safety:

QoS

56Gb/s per port

Regulatory Compliance

EMC



(Emissions): US/Canada: cTUVus certificate

International: CB report and

 certificate

Russia: GOST-R certificate

Argentina: S-mark certificate

DoC

CE EMC & safety report and certificate

USA: FCC part 15 class A

Canada: ICES-003 class A

Australia / New Zealand:



AS/NZS CISPR 22 class A

Japan: VCCI, Class A

Australia C-Tick certificate BSMI certificate (for the power supply only)

Environmental /

Vibration /

Environmental:

EU: IEC 60068-2-64: Random

Vibration:

EU: IEC 60068-2-29: Shocks,

Type I / II

EU: IEC 60068-2-32: Fall Test

Acoustic /

Shock and

Vibration:

Acoustic: 76 DbA

ISO 7779

ETS 300 753

Shock and

Vibration: ETSI EN 300 019-2-2: 1999-09


InfiniBand FDR Switch Platform Hardware User Manual Rev 1.5

Leafs:

Spines:

Switching

Performance /

Capacity

Addressing:

Switching



Capacity:

Scalability and Performance

Up to 12 leafs; 18-ports each

6 spine boards

All spines are needed for nonblocking configuration

Power Supplies:

Management

Modules:

4 Power Supplies standard

2 available 1 required for operation

48K Unicast Addresses Max. per

Subnet

16K Multicast Addresses per

Subnet

Leaf fans:

24.2 Tb/s

4

Switching



Performance: Simultaneous wire-speed any port to any port

A.1

EMI Certification

EMI certification on the fully populated chassis was performed with the chassis installed in a closed two-door rack using the chassis installation kit supplied by Mellanox Technologies.

A.2

Approved Cables

For a list of all approved cables see: http://www.mellanox.com/related-docs/user_manuals/Mellanox_approved_cables.pdf

A.3

EMC Certifications

The list of approved certifications per chassis in different regions of the world is located on the Mellanox Website at: http://www.mellanox.com/related-docs/user_manuals/

Regulatory_and_Compliance_Guide.pdf

EMC Statements are also in the Regulatory and Compliance Guide.


Rev 1.5

Appendix B: Thermal Threshold Definitions

There are three thermal threshold definitions for the SwitchX® switch device which impact the overall switch system operation state: Warning, Critical and Emergency.

1. Warning – 100 o

C

On managed systems only: When the SwitchX® device crosses the 100 o

C threshold, a Warning

Threshold message will be issued by the MLNX-OS management SW, indicating to system administration that the switch has crossed the Warning threshold.

Note that this temperature threshold does not require nor lead to any action by hardware (such as switch shutdown).

2. Critical – 120 o

C

When the SwitchX® device crosses this temperature, the firmware will automatically shut down the device.

3. Emergency – 130 o

C

In case the firmware fails to shut down the SwitchX® device upon crossing the Critical threshold, the SwitchX® device will auto-shutdown upon crossing the Emergency (130 o

C) threshold.



Appendix C: Calculating the Weight of a Customized

Chassis

The weight of a customized chassis can be calculated for any possible customization as follows.

Take the weight of a chassis with the following FRUs installed.

• All fans modules

• All power supplies

The weight of the SX6512 chassis configured above is kg.

To this add the weight of installed FRUs.

• Spines

• Leafs

• Leaf blanks


Fill in the table below to calculate the weight of your system.

Rev 1.5

Number of FRUs FRU Type

Chassis as shipped

Weight of 1 FRU Total Weight / FRU Type

# of Spines * 3.77 =

# of leafs * 2.76 =

# of leaf blanks * 0.764 =

# of management modules * 3.15 =

Total =

This total is in kilograms. Multiply the total by 2.2 to get the total weight in pounds.


Rev 1.5

Appendix D: Calculating the Power of a Chassis

To calculate the power consumption of a chassis take the base power and add the power of the fans, spines, management modules, and leafs.

Table 15 - Power Consumption of Chassis Parts

Part

IB FDR leaf

Spine

Fan

Management Module

Typ Power Passive Cables

Typ Power Active Cables @ 2w Per

Port

91W

108W

117W

14W

125

Not applicable

Not applicable

Not applicable



Appendix E: QSFP Interface

31

32

33

34

35

36

37

38

20

21

22

23

24

25

26

27

28

29

30

J3 and J6

GND

Rx2n

Rx2p

GND

Rx4n

Rx4p

GND

ModPrsL

IntL

VccTx

Vcc1

LPMode

GND

Tx3p

Tx3n

GND

Tx1p

Tx1n

GND

GND

Rx1n

Rx1p

GND

Rx3n

Rx3p

GND

SDA

SCL

Vcc Rx

ResetL

ModSelL

GND

Tx4p

Tx4n

GND

Tx2p

Tx2n

GND

9

8

7

3

2

1

6

5

4

12

11

10

19

18

17

16

15

14

13

Table 16 - InfiniBand QSFP Connector Pino

10

11

12

13

7

8

9

30

31

32

26

27

28

29

Connector

Pin

Number

Connecto r Pin

Name

1

2

3

4

5

6

14

15

16

17

18

19

20

21

22

23

24

25

GND Ground

Signal Description

Tx2n Transmitter Inverted Data Inp

Tx2p Transmitter Non-Inverted Da

Input

GND Ground

Tx4n Transmitter Inverted Data Inp


Input

GND Ground

ModSelL Module Select

ResetL Module Reset

Vcc Rx +3.3 V Power supply receive

SCL 2-wire serial interface clock

SDA 2-wire serial interface data

GND Ground

Rx3p Receiver Non-Inverted Data put

Rx3n Receiver Inverted Data Outpu

GND Ground

Rx1p Receiver Non-Inverted Data put


GND Ground

GND Ground


Rx2p Receiver Non-Inverted Data put 3

GND Ground


Rx4p Receiver Non-Inverted Data put 3

GND Ground

ModPrsL Module Present

IntL Interrupt

Vcc Tx +3.3 V Power supply transmi

Vcc 1 +3.3 V Power Supply

LPMode Low Power Mode

GND Ground


Rev 1.5


Rev 1.5

Figure 52: QSFP Connector Male and Female Views

View into Rear of

Connector

Top

18.35

8.50

View into Front of Cage

8.50

Top

18.35



Appendix F: Replacement Parts Ordering Numbers

Table 17 - Replacement Parts Ordering Numbers

Part Description OPN

Power supply unit PSU 1000W for Modular Switch

Family

MTP005001

Power Supply unit PSU 1600W for Modular Switch

Family

MTP006002

Power supply blank

Leaf board unit 18 port FDR

Leaf board unit 18 port FDR-10

Modular Switch Family, Leaf - Blank

MTM005001

MSX6001FR

MSX6001TR

MTM005004

Spine board unit for FDR system

Spine board unit for FDR10/QDR system

Spine board blank

PPC460 Management Module

MSX6002FLR

MSX6002TBR

MTM005002

MSX6000MAR

x86 Management Module

Modular Switch Family, Management - Blank

MSX6000MBR

MTM005003

Rack Installation Kit for IS5600/SX6536 Series MTR005600

Cable Management Device Set for IS5600/SX6536 MTR005601

Cables Set For IS5600/SX6536 Series MTR005602

IS5600/SX6536, IS5300/SX6518 Series Modular

Switch, leaf cooling chassis fan unit located on Leaf

Side

MTF005001

IS5600/SX6536, IS5300/SX6518 Modular Switch, leaf cooling chassis fan unit located on Spine Side

SX6536, SX6518, MSX6512,MSX 6506

Modular

Switch Series Spine Fan Unit

DB9 to RJ45 Harness

MTF005002

MTF005005

Power cord 250V 15A 2.0M C14 to C13

Power cord 125V 15A 2.0M C14 TO C13

HAR000028

ACC000334

ACC000242

Power cord Type B for USA, Canada, Mexico, Taiwan ACC000204

Power cord Type H for Israel ACC000205

Power cord Type E/F for Sweden, France, Germany,

Netherlands, Russia

ACC000207


Rev 1.5

85

Rev 1.5

Table 17 - Replacement Parts Ordering Numbers (Continued)

Part Description

Power cord Type G for UK

Power cord Type D for India

Power cord Type I for China

Power cord Type J for Switzerland

Power cord Type B for Japan

Power cord Type I for Australia

OPN

ACC000208

ACC000209

ACC000210

ACC000211

ACC000212

ACC000213



Appendix G: Avertissements de sécurité d’installation

(French)

1. Instructions d’installation

Rev 1.5

Lisez toutes les instructions d’installation avant de brancher le matériel à la source d’alimentation électrique.

2. Température excessive

Ce matériel ne doit pas fonctionner dans une zone avec une température ambiante dépassant le maximum recommandé de 45°C (113°F). Un flux d’air de 200LFM à cette température ambiante maximale est nécessaire. En outre, pour garantir un bon

écoulement de l’air, laissez au moins 8 cm (3 pouces) d’espace libre autour des ouvertures de ventilation.

3. Empilage du châssis

Le châssis ne doit pas être empilé sur un autre matériel. Si le châssis tombe, il peut provoquer des blessures corporelles et des dégradations de biens.

4. Connection d'Alimentation electrique excedentaire -dangers électriques

Ce produit comporte un couvercle transparent sur l’espace pour l’alimentation électrique redondante.



Ne pas faire fonctionner le produit si le couvercle transparent n’est pas solidement fixé ou s’il est enlevé.

5. Système de fusible neutre/à double pôle

Avertissement: Système de fusible neutre/à double pôle. Veuillez debrancher tous les cordons d'alimentation avant d'ouvrir le boitier de ce produit ou de toucher un de ses composants internes.

6. Orages – dangers électriques

Pendant un orage, il ne faut pas utiliser le matériel et il ne faut pas brancher ou débrancher les câbles.

7. Branchement/débranchement des câbles InfiniBand en cuivre

Les câbles InfiniBand en cuivre sont lourds et ne sont pas flexibles, il faut donc faire très attention en les branchant et en les débranchant des connecteurs. Consultez le fabricant des câbles pour connaître les mises en garde et les instructions spéciales.


Rev 1.5

8. Risque de choc et de danger

Risque de choc et de danger

 e’lectriques. Le de’branchment d’une seule alimentation stabilise’e ne de’branch uniquement qu’un module “Alimentation Stabilise’e”. Pour isoler completement le module en cause, Il faut de’brancher toutes les alimentations stabilise’es




9. Montage et entretien sur baie

Lorsque ce produit est monté ou entretenu sur baie, il faut prendre des précautions spéciales pour s’assurer que le système reste stable. En général, il faut remplir la baie avec du matériel de bas en haut.

10. Fuite>3.5mA Leakage >3.5mA

« ATTENTION – La connexion à la terre des forts courants de fuite est essentielle avant le branchement de l’alimentation. »

Avant de brancher l’appareil à la conduite d’alimentation, les vis de protection à la terre du terminal de l’appareil doivent être appliquées à l’installation de protection à la

Terre du bâtiment.

11. Forts Courants de Fuite High Leakage Current

Attention: Forts courants de fuite. Il est essentiel de relier a la terre avant de brancher l'alimentation.

12. Ajouter une information de connexion à la masse Connect a Valid Ground to this Device

Avant de brancher l’appareil à la conduite d’alimentation, les vis de protection à la terre du terminal de l’appareil doivent être appliquées à l’installation de protection à la

Terre du bâtiment.

13. Installation du matériel

Ce matériel ne doit être installé, remplacé ou entretenu que par du personnel formé et qualifié.

14. Elimination du matériel

L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur.



15. Codes électriques locaux et nationaux

Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux.

Rev 1.5

16. Codes d’installation

L’appareil doit être installé selon l’ancienne version des codes électriques nationaux du pays. Pour l’Amérique du Nord, l’équipement doit être installé conformément aux spécifications du Code Electrique National Américain et du Code Electrique Canadien.

17. Interconnexion des unites

Les câbles de branchement à l’unité RS232 et les interfaces Ethernet doivent être certifiés UL de type DP-1 ou DP-2. (Note - lorsqu’il existe dans un circuit non LPS)

Protection contre la surintensité : Un appareil de protection répertorié facilement accessible contre la surintensité du circuit de branchement et calibré à 20A doit être incorporé dans le câblage électrique du bâtiment.

18. Exposition au rayonnement grave

Mise en garde – l'utilisation de commandes ou de réglages ou l'exécution de procédures autres que ce qui est spécifié dans les présentes peut engendrer une exposition au rayonnement grave.

PRODUIT LASER DE CLASSE 1 » et références aux normes laser les plus récentes

CEI 60 825-1:1993 + A1:1997 + A2:2001 et NE 60825-1:1994+A1:1996+ A2:2001

19. S’assurer que les enceintes sont appropriées

Des enceintes électriques, mécaniques et incendie adaptées doivent être fournies par le

 fabricant du produit final ou par l’utilisateur final.

20. Cordons électriques CA homologués UL

Pour les prises électriques en Amérique du Nord, choisissez un cordon électrique homologué UL et certifié CSA

à 3 conducteurs, [18 AWG], terminé par une fiche moulée, d’une tension nominale de

125 V, [15 A], avec une longueur minimale de 1,5 m [6 pieds] et d’une longueur maximale de 4,5 m [18 pieds]

Pour les prises électriques en Europe, choisissez un cordon électrique harmonisé internationalement et marqué "<HAR>",

à 3 conducteurs, d’un diamètre de fil minimum de 0,75 mm2, d’une tension nominale de 300 V, avec une gaine isolée en PVC. Le cordon doit avoir une fiche moulée d’une tension nominale de 250 V et d’une intensité nominale de 10 A.


Rev 1.5

21.

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

70 - 121 lbs

32 - 55 kgs

>121 lbs

>55 kgs



Appendix H: Installation - Sicherheitshinweise (German)

1. Installationsanleitungen

Rev 1.5

Lesen Sie alle Installationsanleitungen, bevor Sie das Gerät an die Stromversorgung anschließen.

2. Übertemperatur

Dieses Gerät sollte nicht in einem Bereich mit einer Umgebungstemperatur über der maximal empfohlenen Temperatur von 45°C (113°F) betrieben werden. Es ist ein Luftstrom von 200 LFM bei maximaler Umgebungstemperatur erforderlich. Außerdem sollten mindestens 8 cm (3 in.) Freiraum um die Belüftungsöffnungen sein, um einen einwandfreien Luftstrom zu gewährleisten.

3. Stapeln des Chassis

Das Chassis sollte nicht auf andere Geräte gestapelt werden. Wenn das Chassis herunterfällt, kann es zu Verletzungen und Beschädigungen an Geräten führen.

4. Redundanter Stromversorgungsanschluss - Elektrische Gefahr

Dieses Produkt verfügt über eine Abdeckung über dem Bereich für die redundante

Stromversorgung. Betreiben Sie das Produkt nicht, wenn diese Abdeckung nicht sicher festsitzt oder entfernt wurde.

5. Bei Gewitter - Elektrische Gefahr

Arbeiten Sie während eines Gewitters und Blitzschlag nicht am Gerät, schließen Sie keine Kabel an oder ab.

6. Anschließen/Trennen von InfiniBand-Kupferkabel

InfiniBand-Kupferkabel sind schwer und nicht flexible. Deshalb müssen sie vorsichtig an die Anschlüsse angebracht bzw. davon getrennt werden. Lesen Sie die speziellen

Warnungen und Anleitungen des Kabelherstellers.

7. Gafahr des elektrischen Schocks.

Gafahr des elektrischen Schocks. Entferrnen des Netzsteckers elnes Netzteils spannungsfrei. Um alle Einhieten spannungsfrei zu machen sind die Netzstecker aller Netzteile zu entfernen


Rev 1.5




8. Rack-Montage und Wartung

Wenn dieses Produkt in einem Rack montiert oder gewartet wird, sind besondere Vorsichtsmaßnahmen zu ergreifen, um die Stabilität des Systems zu gewährleisten. Im

Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen.

9. Geräteinstallation

Diese Gerät sollte nur von geschultem und qualifiziertem Personal installiert, ausgetauscht oder gewartet werden.

10. Geräteentsorgung

Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestimmungen erfolgen.

11. Regionale und nationale elektrische Bestimmungen

Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen

Bestimmungen installiert werden.

12. Richtigen Schutz sicherstellen

Geeigneter elekrischer, mechanischer und Feuerschutz sind vom



Hersteller des Endprodukts oder dem Endbenutzer bereitzustellen.

13. Strahlenkontakt

Achtung – Nutzung von Steuerungen oder Einstellungen oder Ausführung von Prozeduren, die hier nicht spezifiziert sind, kann zu gefährlichem Strahlenkontakt führen..

Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards :



ICE 60 825-1:1993 + A1:1997 + A2:2001 und EN 60825-1:1994+A1:1996+ A2:2001



14. UL-und CSA Certified Netzkabel

Für Nordamerika Stromanschluss, wählen Sie ein Netzkabel, das UL-und CSA Certified

3 - Leiter, [18 AWG], mit einem angespritztem Stecker bewertet bei 125 V, [15], mit einer Mindestlänge von 1,5 m [Six Feet] aber nicht mehr als 4,5 m.

Für die europäischen Zusammenhang, wählen Sie ein Netzkabel, das international harmonisiert und der Aufschrift "<HAR>",

3 - Leiter, mindestens 0,75 mm2 Draht, bewertet mit 300 V, mit einem PVC-Mantel isoliert. Das Kabel muss eine angespritztem Stecker bewertet bei 250 V, 10 A. "

Rev 1.5

15. Ableitstrom> 3.5mA LEAKAGE >3.5mA

WARNUNG: Hohe Ableitstrom; Earth Verbindung, bevor Sie die Verbindung von wesentlicher Bedeutung werden.

16. Add GND Verbindung Informationen

Bevor Sie dieses Gerät an das Stromnetz, die Schutzerde Terminal Schrauben dieses

Gerät muss an den Schutzleiter in der Gebäudeinstallation.

17. Installation Codes

Dieses Gerät muss installiert sein, entsprechend auf die neueste Version des Landes

National Electrical Code. Für Nordamerika, müssen in Übereinstimmung mit den geltenden Vorschriften in der US-amerikanischen National Electrical Code und dem

Canadian Electrical Code.

18. Zusammenschaltung von EINHEITEN

Kabel für den Anschluss an das Gerät RS232-und Ethernet-Schnittstellen müssen UL zertifiziert Typ DP-1 oder DP-2. (Hinweis-, wenn nicht mit Wohnsitz in LPS-Schaltung)

Überstromschutz: Eine leicht zugängliche Auflistung Abzweigleitung Überstrom-

Schutzeinrichtung 20 A bewertet werden müssen in dem Gebäude Verkabelung.

19. Zweipolige bzw. Neutralleiter-Sicherung im Netzteil

Achtung:

Zweipolige bzw. Neutralleiter-Sicherung im Netzteil. Netzstecker ziehen, um sicherzustellen, daß keine Spannung am Gerät anliegt. Entfernen Sie alle Netzkabel vor dem

Öffnen der Abdeckung dieses Produkts oder dem Berühren der Innenteile.


Rev 1.5

20.

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

70 - 121 lbs

32 - 55 kgs

>121 lbs

>55 kgs



Appendix I: Advertencias de seguridad para la instalación (Spanish)

1. Instrucciones de instalación

Rev 1.5

Antes de conectar el equipo a la fuente de alimentación, leer todas las instrucciones de instalación.

2. Lesión corporal por peso

<40 lbs

<18 kgs

40 - 70 lbs

18 - 32 kgs

3. Lesión corporal por peso

70 - 121 lbs

32 - 55 kgs

>121 lbs

>55 kgs

This equipment is very heavy and should be moved using a mechanical lift to avoid injuries.

4. Sobrecalentamiento

No se debe utilizar el equipo en un área con una temperatura ambiente superior a la máxima recomendada: 45°C. Además, para garantizar una circulación de aire adecuada, se debe dejar como mínimo un espacio de 8 cm (3 pulgadas) alrededor de las aberturas de ventilación.

5. Apilamiento del chasis

Los chasis no se deben apilar sobre otros equipos. La caída del chasis podría causar lesiones corporales, así como daños al equipo.

6. Dos fusibles, uno en el polo y otro en el neutro

Dos fusibles, uno en el polo y otro en el neutro. Quitar los cables de corriente antes de abrir la tapa de este producto o tocar cualquier componente interno.


Rev 1.5

7. Cuando hay rayos: peligro de descarga eléctrica

No utilizar el equipo ni conectar o desconectar cables durante períodos de actividad de rayos.

8. Conexión y desconexión del cable Copper InfiniBand

Dado que los cables de cobre InfiniBand son pesados y no son flexibles, su conexión a los conectores y su desconexión se deben efectuar con mucho cuidado. Para ver advertencias o instrucciones especiales, consultar al fabricante del cable.

9. Montaje y mantenimiento de bastidores

Al instalar o realizar el mantenimiento de este aparato en un bastidor, es preciso adoptar precauciones especiales para garantizar que el sistema se mantenga estable. En general, en un bastidor, los equipos se deben instalar comenzando desde abajo hacia arriba.

10. Instalación de equipos

La instalación, el reemplazo y el mantenimiento de este equipo estarán a cargo únicamente de personal capacitado y competente.

11. Asegurar confinamientos adecuados

El fabricante del producto final o el usuario final deberán suministrar

 un confinamiento adecuado para componentes eléctricos y mecánicos y contra incendio.

12. Eliminación de equipos

La eliminación definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales.

13. Códigos eléctricos locales y nacionales

Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales.



14.

Cable de alimentación homologado por UL y con certificación CSA

En conexiones de América del Norte, seleccionar un cable de alimentación homologado por UL y con certificación CSA de tres conductores, [16 AWG], terminado en un enchufe moldeado con capuchón de 125 voltios nominal, [13 A], con una longitud mínima de 1,5 metros, pero no más de 4,5 metros.

En conexiones europeas, seleccionar un cable de alimentación armonizado

 internacionalmente y marcado "<HAR>", de tres conductores, hilo de 1,0 mm2 como mínimo, 300 voltios nominal, con cobertura protectora aislante de PVC. El cable debe tener un enchufe moldeado con capuchón de 250 voltios nominal, 10 A.

15. Añadir conexión a tierra

Antes de conectar el dispositivo a la línea de alimentación, los tornillos del terminal de la puesta a tierra de protección del dispositivo se deben conectar a la puesta a tierra de protección de la instalación del edificio.

(Información de conexión a tierra):

La instalación del edificio deberá prover un medio para la conexión con la puesta a tierra de protección y un técnico de servicio deberá conectar permanentemente el equipo a dicho medio de conexión.

Un TÉCNICO DE SERVICIO comprobará si la toma eléctrica de la que se suministrará corriente al equipo provee una conexión con la puesta a tierra de protección del edificio. De no ser así, el TÉCNICO DE SERVICIO se encargará de instalar un CON-

DUCTOR DE CONEXIÓN A TIERRA DE PROTECCIÓN, del terminal de puesta a tierra de protección separado al conductor de tierra de protección del edificio. El equipo se instalará en un área donde haya conexión equipotencial, como por ejemplo, un centro de telecomunicaciones o una sala de computadoras dedicada.

16. Códigos de instalación

Este dispositivo se debe instalar conforme a la versión más reciente de los códigos eléctricos nacionales del país en cuestión. En América del Norte, el equipo se debe instalar de acuerdo con las disposiciones vigentes del Código Eléctrico Nacional de los

EE.UU. y del Código Eléctrico de Canadá.

17. Directiva sobre RAEE

Rev 1.5

Conforme a la Directiva 2002/96/CE sobre RAEE, todos los residuos de equipos eléctricos y electrónicos (EEE) se deben recolectar por separado y no se deben eliminar junto con residuos domésticos.

Al deshacerse de este producto y de todas sus partes, hágalo de una manera responsable y respetuosa con el medio ambiente.


No results