- No category
Artesyn ATCA-9405-Installation and Use
Add to my manualsadvertisement
ATCA-9405
Installation and Use
P/N: 6806800M71F
November 2013
Embedded Computing for
Business-Critical Continuity
TM
©
2013 Emerson
All rights reserved.
Trademarks
Emerson, Business-Critical Continuity, Emerson Network Power and the Emerson Network Power logo are trademarks and service marks of Emerson Electric Co.
©
2013 Emerson Electric Co. All other product or service names are the property of their respective owners.
Intel
®
is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.
Java
™
and all other Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries.
Microsoft
®
, Windows
®
and Windows Me
®
are registered trademarks of Microsoft Corporation; and Windows XP
™
is a trademark of
Microsoft Corporation.
PICMG
®
, CompactPCI
®
, AdvancedTCA
™
and the PICMG, CompactPCI and AdvancedTCA logos are registered trademarks of the PCI
Industrial Computer Manufacturers Group.
UNIX
®
is a registered trademark of The Open Group in the United States and other countries.
Notice
While reasonable efforts have been made to assure the accuracy of this document, Emerson assumes no liability resulting from any omissions in this document, or from the use of the information obtained therein. Emerson reserves the right to revise this document and to make changes from time to time in the content hereof without obligation of Emerson to notify any person of such revision or changes.
Electronic versions of this material may be read online, downloaded for personal use, or referenced in another document as a URL to a Emerson website. The text itself may not be published commercially in print or electronic form, edited, translated, or otherwise altered without the permission of Emerson,
It is possible that this publication may contain reference to or information about Emerson products (machines and programs), programming, or services that are not available in your country. Such references or information must not be construed to mean that
Emerson intends to announce such Emerson products, programming, or services in your country.
Limited and Restricted Rights Legend
If the documentation contained herein is supplied, directly or indirectly, to the U.S. Government, the following notice shall apply unless otherwise agreed to in writing by Emerson.
Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (b)(3) of the Rights in
Technical Data clause at DFARS 252.227-7013 (Nov. 1995) and of the Rights in Noncommercial Computer Software and
Documentation clause at DFARS 252.227-7014 (Jun. 1995).
Contact Address
Emerson Network Power - Embedded Computing
Lilienthalstr. 17 - 19
85579 Neubiberg/Munich
Germany
Regulatory Agency Warnings & Notices
The Emerson ATCA-9405 meets the requirements set forth by the Federal Communications
Commission (FCC) in Title 47 of the Code of Federal Regulations. The following information is provided as required by this agency.
This device complies with part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) This device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.
FCC Rules and Regulations – Part 15
This equipment has been tested and found to comply with the limits for a Class A Class B digital device, pursuant to part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed and used in accordance with the instructions, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at his own expense.
Making changes or modifications to the ATCA-9405 hardware without the explicit consent of
Emerson Network Power could invalidate the user’s authority to operate this equipment.
EMC Compliance
The electromagnetic compatibility (EMC) tests used an ATCA-9405 model that includes a front panel assembly from Emerson Network Power.
ATCA-9405 Installation and Use (6806800M71F) 3
Regulatory Agency Warnings & Notices
GR-1089-CORE STANDARD
The intra-building port(s) of the equipment or subassembly is suitable for connection to intrabuilding or unexposed wiring or cabling only. The intra-building port(s) of the equipment or subassembly MUST NOT be metallically connected to interfaces that connect to the OSP or its wiring. These interfaces are designed for use as intra-building interfaces only (Type 2 or Type 4 ports as described in GR-1089-CORE, Issue 4) and require isolation from the exposed OSP cabling. The addition of Primary Protectors is not sufficient protection in order to connect these interfaces metallically to OSP wiring.
4 ATCA-9405 Installation and Use (6806800M71F)
ATCA-9405 Installation and Use (6806800M71F) 5
6 ATCA-9405 Installation and Use (6806800M71F)
Contents
ATCA-9405 Installation and Use (6806800M71F) 7
8 ATCA-9405 Installation and Use (6806800M71F)
Contents
ATCA-9405 Installation and Use (6806800M71F) 9
10 ATCA-9405 Installation and Use (6806800M71F)
List of Tables
ATCA-9405 Installation and Use (6806800M71F)
11
List of Tables
Emerson Network Power - Embedded Computing Publications . . . . . . . . . . . . . . . . . .153
12 ATCA-9405 Installation and Use (6806800M71F)
List of Figures
Fabric Interface Mode 1x 1000Base-KX (PICMG3.1 Option 1) . . . . . . . . . . . . . . . . . 95
Fabric Interface Mode 1x 10GBase-KX4 (PICMG3.1 Option 9) . . . . . . . . . . . . . . . . . 95
ATCA-9405 Installation and Use (6806800M71F) 13
List of Figures
14 ATCA-9405 Installation and Use (6806800M71F)
About this Manual
Overview of Contents
This manual is divided into the following chapters and appendix:
Chapter 1, Introduction, on page 21
, provides an overview of the ATCA-9405 board.
Chapter 2, Setup, on page 29 , describes the ATCA-9405 board dimensions, power
requirements, environmental requirements, how to insert and remove the board, troubleshooting and so on.
Chapter 3, Packet Processor, on page 57 , provides an overview of the CN6880 Processor and
its units.
Chapter 4, Service Processor, on page 69 , provides an overview of the P2020 Processor and
its units.
Chapter 5, Ethernet Infrastructure, on page 87 , describes the ethernet switch, base interface,
fabric interface, update channel interface, and the serial redirection.
Chapter 6, Service Infrastructure, on page 99
, describes the PCI express switch, hot plug support, I2C slave interface, JTAG support, and lane status.
Chapter 7, Mezzanine Module, on page 105 , provides an overview.
Chapter 8, Intelligent Peripheral Management Controller, on page 109 , provides a functional
overview, firmware architecture, HPM.1 components, Sensors, and so on.
Appendix A, Related Documentation, on page 153 , lists the documents that you can refer.
Safety Notes on page 155 , provides the safety measures to be taken while handling the
board.
Sicherheitshinweise on page 159 , provides the German translation of the Safety Notes.
ATCA-9405 Installation and Use (6806800M71F) 15
Abbreviations
This document uses the following abbreviations:
IPMB
IPMC
IPMI
JTAG
L2
LCCB
LED
DRAM
DXAUI
ECC
EEPROM
EHCI
EMC
ESD
FRU
GPIO
I/O
I
2
C
IMC
Abbreviation
ATCA
BIOS
CPLD
DDR
DDR3
DMA
Definition
Advanced Telecom Computing Architecture
Basic Input/Output System
Complex Programmable Logic Device
Dual Data Rate
Double Data Rate 3
Direct Memory Access
Dynamic Random Access Memory
Double Data Rate XAUI
Error Correction Code
Electrically Erasable Programmable Read Only Memory
Enhanced Host Controller Interface
Electro-magnetic Compatibility
Electro-static Discharge
Field Replaceable Unit
General Purpose Input/Output
Input/Output
Inter Integrated-Circuit Bus (2-wire serial bus and protocol)
Integrated Memory Controller
Intelligent Platform Management Bus
Intelligent Platform Management Controller
Intelligent Platform Management Interface
Joint Test Action Group (test interface for digital logic circuits)
Level 2
Line Card Clock Building Block
Light Emitting Diode
16 ATCA-9405 Installation and Use (6806800M71F)
About this Manual
SELV
SerDes
SGMII
SMP
SPD
TCAM
RoHS
RTC
RTM
Rx
RXAUI
SDR
SDRAM
SECDED
OEM
OOS
PCB
PCIe
PHY
PICMG
PLL
POST
Abbreviation
LPC
MAC
MMC
MTBF
NEBS
NVRAM
ATCA-9405 Installation and Use (6806800M71F)
Definition
Low Pin Count
Medium Access Controller
Module Management Controller
Mean Time Between Failures
Network Equipment Building System
Non-volatile Random Access Memory
Original Equipment Manufacturer
Out-of-service
Printed Circuit Board
PCI-Express
Physical layer device (for Ethernet)
PCI Industrial Computer Manufacturers Group
Phase Locked Loop
Power-on Self Test
Restriction of Hazardous Substances
Real-Time Clock
Rear Transition Module
Receive line (of a duplex serial communication interface)
Reduced XAUI
Sensor Data Record
Synchronous Dynamic Random Access Memory
Single Error Correction Double Error Detection
Safety Extra Low Voltage
Serializer/Deserializer
Serial Gigabit Media Independent Interface
Symmetric Multi-Processors
Serial Presence Detect
Ternary Content Addressable Memory
17
Abbreviation
TCP
Tx
UART
USB
UTMI
XAUI
XLAUI
Definition
Transmission Control Protocol
Transmit line (of a duplex serial communication interface)
Universal Asynchronous Receiver-Transmitter
Universal Serial Bus
USB 2.0 Transceiver Macrocell Interface
10 Gigabit Attachment Unit Interface
40 Gigabit Attachment Unit Interface
Conventions
The following table describes the conventions used throughout this manual.
Notation
0x00000000
0b0000 bold
Screen
Courier + Bold
Reference
File > Exit
<text>
[text]
...
Description
Typical notation for hexadecimal numbers (digits are
0 through F), for example used for addresses and offsets
Same for binary numbers (digits are 0 and 1)
Used to emphasize a word
Used for on-screen output and code related elements or commands in body text
Used to characterize user input and to separate it from system output
Used for references and for table and figure descriptions
Notation for selecting a submenu
Notation for variables and keys
Notation for software buttons to click on the screen and parameter description
Repeated item for example node 1, node 2, ..., node
12
18 ATCA-9405 Installation and Use (6806800M71F)
About this Manual
Notation
.
.
.
..
|
Description
Omission of information from example/command that is not necessary at the time being
Ranges, for example: 0..4 means one of the integers
0,1,2,3, and 4 (used in registers)
Logical OR
Indicates a hazardous situation which, if not avoided, could result in death or serious injury
Indicates a hazardous situation which, if not avoided, may result in minor or moderate injury
Indicates a property damage message
No danger encountered. Pay attention to important information
Summary of Changes
This manual has been revised and replaces all prior editions.
Part Number Publication Date
6806800M71A September 2011
6806800M71B November 2011
Description
Preliminary copy
DA version
ATCA-9405 Installation and Use (6806800M71F) 19
Part Number Publication Date
6806800M71C June, 2012
6806800M71D December, 2012
6806800M71E May, 2013
6806800M71F November, 2013
Description
EA version
GA version
,
Establishing a Telnet Session on page
, and IPMI command examples in
Updated Chapter 8, Establishing a Telnet Session, on page 140
,
and
.
20 ATCA-9405 Installation and Use (6806800M71F)
Chapter 1
Introduction
1.1
Overview
The ATCA-9405 is an Advanced Telecom Computing Architecture (AdvancedTCA ® , ATCA ® ) blade based on two Cavium OCTEON II CN6880 Multi-Core MIPS64 Packet Processors and the dual-core Freescale P2020 Service Processor.
The MIPS64 processor cores can be configured for up to 32-way SMP LINUX support, run
Cavium Simple Executives, and run fast path packet processing software for time critical applications. The ATCA-9405 supports for up to 162 Gigabit Ethernet (GbE) I/O bandwidth and a redundant 40GbE fabric. It allows the design of compact application systems for the upcoming new IP data infrastructure.The ATCA-9405 represents a performance and throughput increase over previous generations.
The dual-core Free scale QorIQ™ P2020 processor is used for basic board setup, general board management, and Ethernet switch management. The management processor has local mass storage support for uboot and user code. The boot code for the OCTEON II packet processors is provided via the management processor. The dual-core processor is used to offload other blade functions in order to maximize the packet processing capability, including managing
Layer 2 and 3 switching/routing functions on the local Ethernet switch.
This blade is targeted at security and packet-processing applications in the wireless and transport market segments. These markets include dataplane packet-processor, security coprocessor, video compression, and pattern matching.
The ATCA-9405 complies with the SCOPE recommended profile for central office ATCA systems, PICMG ® 3.0 ATCA mechanical specifications, E-keying, and Hot Swap.
ATCA-9405 Installation and Use (6806800M71F) 21
Introduction
1.2
Components and Features
The ATCA-9405 hardware components and its features are:
Single-slot ATCA form factor
Two Cavium OCTEON II CN6880 Multi-Core MIPS64 Packet Processors each with memory support up to 64 GB DDR3
Dual Core Free scale P2020 Service Processor for blade management with up to 8 GB DDR3
Memory
Marvell 98CX8234 32-port Ethernet switch connecting all rear I/O, backplane I/O and
Packet Processors with L2 and L3 switch management software.
Dual 1G/10G/40G Ethernet fabric interface
Dual 1G Ethernet base interface
Support for Wind River PNE 4.x OS, Cavium SDK, and 6WIND 6WINDGate
22 ATCA-9405 Installation and Use (6806800M71F)
1.3
Functional Overview
The following block diagram provides a functional overview of the ATCA-9405.
Figure 1-1 General System Block Diagram
Introduction
ATCA-9405 Installation and Use (6806800M71F) 23
Introduction
1.4
Additional Information
This section lists the regulatory certifications of ATCA-9405 hardware and briefly discusses the terminology and notation conventions used in this manual.
Mean time between failures (MTBF) has been calculated at 439,924 hours using the Telcordia
SR-332, Issue 1 (Reliability Prediction for Electronic Equipment), method 2 at 30°C.
1.4.1
Regulatory Compliance
The ATCA-9405 has been tested to comply with various standards:
Table 1-1 Regulatory Compliance
Item
Designed to comply with NEBS,
Level 3
Designed to comply with ETSI
Description
Telcordia GR-63-CORE, NEBS Physical Protection
Telcordia GR-1089-CORE, Electromagnetic Compatibility and
Electrical Safety – Generic Criteria for Network
Telecommunications Equipment. Equipment Type 2
ETSI Storage, EN 300 019-1-1, Class 1.2 equipment, Not
Temperature Controlled Storage Locations
ETSI Transportation, EN 300 019-1-2, Class 2.3 equipment,
Public Transportation
ETSI Operation, EN 300 019-1-3, Class 3.1(E) equipment,
Temperature Controlled Locations
ETSI EN 300 132-2 Environmental Engineering (EE); Power supply interface at the input to telecommunications equipment; Part 2: Operated by direct current (dc)
ETSI ETS 300 753, Equipment Engineering (EE); Acoustic noise emitted by telecommunications equipment
24 ATCA-9405 Installation and Use (6806800M71F)
Introduction
Table 1-1 Regulatory Compliance (continued)
Item
EMC
Safety
RoHS/WEEE compliance
Interoperability
Description
ETSI EN 300 386 Electromagnetic compatibility and Radio spectrum Matters (ERM); telecommunication network equipment; ElectroMagnetic Compatibility (EMC) requirements, Telecommunication equipment room
(attended)
CFR 47 FCC Part 15 Subpart B, Class A (US); FCC Part 15 - Radio
Frequency Devices; Subpart B: Unintentional Radiators
AS/NZS CISPR 22 (Australia/New Zealand), Limits and Methods of Measurement of Radio Disturbance Characteristics of
Information Technology Equipment
VCCI Class A (Japan), Voluntary Control Council for Interference by Information Technology Equipment
CISPR 22 Information technology equipment – Radio disturbance characteristics – Limits and methods of measurement
CISPR 24 Information technology equipment – Immunity characteristics – Limits and methods of measurement
Certified to UL/CSA 60950-1, EN 60950-1 and IEC 60950-1 CB
Scheme
Safety of information technology equipment, including electrical business equipment
DIRECTIVE 2002/95/EC OF THE EUROPEAN PARLIAMENT AND
OF THE COUNCIL on the restriction of the use of certain hazardous substances in electrical and electronic equipment
(RoHS)
DIRECTIVE 2002/96/EC OF THE EUROPEAN PARLIAMENT AND
OF THE COUNCIL on waste electrical and electronic equipment
(WEEE)
Designed to operate within a CP-TA B.4 system environment at full performance
ATCA-9405 Installation and Use (6806800M71F) 25
Introduction
1.4.2
RoHS Compliance
The ATCA-9405 is compliant with the European Union’s RoHS (Restriction of use of Hazardous
Substances) directive created to limit harm to the environment and human health by restricting the use of harmful substances in electrical and electronic equipment. Effective July
1, 2006, RoHS restricts the use of six substances: cadmium (Cd), mercury (Hg), hexavalent chromium (Cr (VI)), polybrominated biphenyls (PBBs), polybrominated diphenyl ethers
(PBDEs), and lead (Pb). Configurations that are RoHS compliant are built with lead-free solder.
To obtain a certificate of conformity (CoC) for the ATCA-9405, visit www.emerson.com/embeddedcomputing
1.4.3
Notation
Active low signals
An active low signal is indicated with an asterisk # after the signal name.
1.5
Ordering Information
Table 1-2 Ordering Information
Part Number
ATCA-9405B-32GB
ATCA-9405B-64GB
ARTM-9405B-16X10GE
Product Description
ATCA-9405 - 2X CN6880-1.2GHZ, 2X 16GB DDR3 MEMORY ON FOUR
LOCAL MEMORY CONTROLLERS - 1X P2020 WITH 2GB MEMORY - 16GB
FLASH - BBS - SWITCH MGMT SW
On customer request
ARTM-9405 - 8X10G (SFP+) and 2x40G (QSFP)
SM-BBS-WR-ATCA-9405 Basic Blade Services software and L2 & L3 switch management software based on Wind River PNE4.x Linux. (CD Media only)
RJ45-DSUB-ATCA7140 RJ-45 DSUB cable for the ATCA-7140, 7150, 7350, 736X
26 ATCA-9405 Installation and Use (6806800M71F)
1.6
Product Identification
Figure 1-2 Serial Number and Product ID
Introduction
On R1.0 prototypes, only the Serial Number label is available.
ATCA-9405 Installation and Use (6806800M71F) 27
Introduction
28 ATCA-9405 Installation and Use (6806800M71F)
Chapter 2
Setup
2.1
Overview
This chapter describes the physical layout of the board, the setup process, and how to check for proper operation once the board has been installed. This chapter also includes troubleshooting, service, and warranty information.
2.2
Electrostatic Discharge
Before you begin the setup process, remember that the electrostatic discharge (ESD) can easily damage the components on the ATCA-9405 hardware. Electronic devices, especially those with programmable parts, are susceptible to ESD, which can result in operational failure.
Unless you ground yourself properly, static charges can accumulate in your body and cause
ESD damage when you touch the board.
Use proper static protection and handle ATCA-9405 boards only when absolutely necessary.
Always wear a wriststrap to ground your body before touching a board. Keep your body grounded while handling the board. Hold the board by its edges—do not touch any components or circuits. When the board is not in an enclosure, store it in a static-shielding bag.
To ground yourself, wear a grounding wriststrap. Simply placing the board on top of a staticshielding bag does not provide any protection—place it on a grounded dissipative mat. Do not place the board on metal or other conductive surfaces.
ATCA-9405 Installation and Use (6806800M71F) 29
Setup
2.3
ATCA-9405 Circuit Board
The ATCA-9405 circuit board is an ATCA blade assembly and complies with the PICMG 3.0 ATCA mechanical specification. It uses a 16-layer printed circuit board with the following dimensions:
Table 2-1 Circuit Board Dimensions
Width Depth Height Weight (typical)
12.687 in.
(322.25 mm)
11.024 in.
(280.01 mm)
< .84 in.
(<21.33 mm)
8.38 lb. (3.80 kg)
This is the typical weight for the ATCA-9405. Board weight varies slightly per configuration; contact
Technical Support if you require a specific configuration weight.
The
, shows the face plate layout for ATCA-9405 board.
Figure 2-1 Faceplate Layout
30
The face plate of the blade provides the following interfaces and control elements.
Table 2-2 Faceplate Interfaces
Processor
Packet Processor
Service Processor
Interfaces / Control Elements
One Serial Console port for Packet Processor 1 (only for maintenance purposes)
One Serial Console port for Packet Processor 2 (only for maintenance purposes)
One 10/100/1000Base-T Ethernet port shared for both Packet
Processors (only for maintenance purposes)
One USB port (only for maintenance purposes)
One Serial Console port (only for maintenance purposes)
One 10/100/1000Base-T Ethernet port (only for maintenance purposes)
ATCA-9405 Installation and Use (6806800M71F)
Setup
Table 2-2 Faceplate Interfaces (continued)
Processor
IPMC
Interfaces / Control Elements
Out of Service (OOS) LED
In Service (IS) LED
Attention (ATN) LED
Hot Swap (H/S) LED
Recessed Reset Button
For these interfaces, the maximum cable length should not exceed more than 3m.
2.3.1
Switch Settings
All mechanical switches are OFF in their default configuration. Switch OFF means high level for the connected signal and switch ON means low level for the connected signal.
ATCA-9405 Installation and Use (6806800M71F) 31
Setup
The switches are placed on component side 1. The Figure 2-2 on page 32
, shows the switch locations on the board.
Figure 2-2 Switch Location with Heat Sink SW1 and SW2
32 ATCA-9405 Installation and Use (6806800M71F)
Figure 2-3 Switch Location with Heat Sink SW3 and SW4
Setup
2.3.1.1
FPGA and CPLD/IPMC Switches
The switch settings are described in Table 2-3 on page 33 .
Table 2-3 Settings for Switch SW1
Switch Description
SW1-1 IPMI Boot Select
OFF = SPI Boot Flash is selected by IPMC
ON = SPI Boot Flash is selected by SW1-2
SW1-2 Manual Boot Select
OFF = Default SPI Boot Flash selected
ON = Recovery SPI Boot Flash selected
OFF
Default is selected by IPMC
SPI Flash
Default SPI Boot Flash is used
ATCA-9405 Installation and Use (6806800M71F) 33
Setup
34
Table 2-3 Settings for Switch SW1 (continued)
Switch Description
SW1-3 Debug Header Boot Select
This switch overrides selection done via
SW1-1 and SW1-2.
OFF = Boot from SPI Boot Flash (TSOP devices)
ON = Boot from SPI Debug Header
SW1-4 Reserved
OFF
OFF
Table 2-4 Settings for Switch SW2 (for debugging only)
Switch
SW2-1 Reserved
SW2-2 Reserved
SW2-3 Reserved
SW2-4 Reserved
Description
OFF
OFF
OFF
OFF
Table 2-5 Settings for Switch SW3
Switch Description
SW3-1 Enable E10-USB Emulator for H8S
OFF = Disabled
ON = Enabled
SW3-2 Enable H8S Programming via Debug Console
OFF = Disabled
ON = Enabled
SW3-3 Manual Payload Power Enable for Blade
OFF = IPMI controlled power enable for blade
ON = Manual power enable for blade
SW3-4 Manual Payload Power Enable for RTM
OFF = IPMI controlled power enable for RTM
ON = Manual power enable for RTM
OFF
OFF
OFF
OFF
Default
Boot from TSOP SPI Flash
Do not change
Default
Do not change
Do not change
Do not change
Do not change
Default
E10-USB Emulator disabled
H8S programming disabled
IPMI controlled power enabled
IPMI controlled power enabled
ATCA-9405 Installation and Use (6806800M71F)
Setup
Table 2-6 Settings for Switch SW4 (for debugging only)
Switch
SW4-1
SW4-2
SW4-3
(dual function)
SW4-4
Description
Manual Power Enable for Packet Processor 1
OFF = SP controlled PP1 power enable
ON = Manual PP1 power enable
Manual Power Enable for Packet Processor 2
OFF = SP controlled PP2 power enable
ON = Manual PP2 power enable
IPMC Watchdog Enable (SW3-3 = OFF)
OFF = Watchdog enabled
ON = Watchdog disabled
Reserved (SW3-3 = ON)
IPMC Console Output Selection
OFF = IPMC console output via debug connector
ON = IPMC console output via front panel connector
Default
OFF Software controlled (SP) power enable for PP1
OFF Software controlled (SP) power enable for PP2
OFF IPMC Watchdog Enabled
OFF Do not change
OFF IPMC Console Output via debugging header
ATCA-9405 Installation and Use (6806800M71F) 35
Setup
2.3.2
Safety Critical Hot Spots
The following figure shows the critical hot spots on the ATCA-9405 board.
Figure 2-4 Location of the Hot Spots
36
Temperature Spot 2 (on 48V DC/DC
Converter) = 100 °C (max) exact location: in the geometric middle of the heat spreader
Temperature Spot 1 (on Power Entry
Module) = 110 °C (max) exact location: on top of upper transformer housing
ATCA-9405 Installation and Use (6806800M71F)
Setup
2.3.3
Connector Pin Assignment
2.3.3.1
Face Plate Connectors
Refer
for the face plate layout.
2.3.3.1.1 USB Port
One standard USB Type-A connector for access to the USB interface of the Service Processor is available at the face plate.
Table 2-7 USB Connector Pin-out
2
3
Pin
1
4
Signal
VCC_USB
D-
D+
GND
2.3.3.1.2
Ethernet Ports
Two shielded RJ45 connectors with integrated transformers for 10/100/1000Base-T Ethernet are available at the front panel.
Table 2-8 Ethernet Connector Pin-out
6
7
4
5
8
2
3
Pin Signal
1 BI_DA+
BI_DA-
BI_DB+
BI_DC+
BI_DC-
BI_DB-
BI_DD+
BI_DD-
ATCA-9405 Installation and Use (6806800M71F) 37
Setup
2.3.3.1.3
Serial Console Ports
Three shielded RJ45 connectors (with Cisco style pin-out) for serial consoles are available at the front panel.
Table 2-9 Serial Console Connector Pin-out (Cisco Style)
5
6
3
4
7
8
Pin
1
2
Signal
RTS#
NC
TXD
GND
GND
RXD
NC
CTS#
38 ATCA-9405 Installation and Use (6806800M71F)
2.3.3.2
On-board Connectors
The following figure shows the location of eUSB Drive Connector (P6) on ATCA-9405.
Figure 2-5 Location of On-board Connectors
Setup
ATCA-9405 Installation and Use (6806800M71F)
P6
39
Setup
2.3.3.2.1
eUSB Drive Connector
ATCA-9405 provides one 2x5 pin header and a mounting hole for eUSB drives like Smart
Modular Z-U130.
Table 2-10 eUSB Drive Connector Pin-out
5
6
3
4
Pin
1
2
7
8
9
10
D-
NC
D+
NC
Signal
VCC
NC
GND
NC
KEY
NC
40 ATCA-9405 Installation and Use (6806800M71F)
Setup
2.3.3.3
Back Panel Connectors
2.3.3.3.1 Overview
The AdvancedTCA backplane connectors reside in three zones, 1 to 3 as specified by the
AdvancedTCA standard and are called P10, P20 and 23, and P30, P31, and 32. The pinouts of all these connectors are given in this section.
Figure 2-6 Location of AdvancedTCA Connectors
ATCA-9405 Installation and Use (6806800M71F) 41
Setup
2.3.3.3.2 Zone 1 Connector
The Zone 1 connector is used to deliver power from the ATCA back plane to the ATCA-9405 baseboard. The pin out for this connector is defined by the ATCA specifications. The
ATCA-9405 does not implement Metallic Test or Ringing Generator Circuitry and therefore does not use pins 17 through 24.
Figure 2-7 P10 Backplane Connector Pinout
42
Table 2-11 Zone 1 Connector P10 Pin Assignment
7
8
9
10
5
6
Contact Destination
1 - 4 Reserved
IPMC ISC PC0
IPMC ISC PC1
11
12
IPMC ISC PC2
IPMC ISC PC3
IPMC ISC PD4
IPMC ISC PD5
IPMC ISC PD6
IPMC ISC PD7
Description
Reserved
Hardware Address Bit 0
Hardware Address Bit 1
Hardware Address Bit 2
Hardware Address Bit 3
Hardware Address Bit 4
Hardware Address Bit 5
Hardware Address Bit 6
Hardware Address Bit 7
ATCA-9405 Installation and Use (6806800M71F)
Setup
Table 2-11 Zone 1 Connector P10 Pin Assignment (continued)
29
30
31
32
25
26
27
28
33
34
Contact Destination
13 IPMC IMC PD0
14
15
16
17-24
IPMC IMC PD1
IPMC ISC PC5
IPMC ISC PC4
Not used
Shelf Ground
Logic Ground
Power Building Block
Power Building Block
Power Building Block
Power Building Block
Power Building Block
Power Building Block
Power Building Block
Power Building Block
Description
IPMB Clock Port A
IPMB Data Port A
IPMB Clock Port B
IPMB Data Port A
Not used
Shelf Ground
Logic Ground
Enable B
Voltage Return A
Voltage Return B
Early -48V A
Early -48V B
Enable A
-48V A
-48V A
2.3.3.3.3 Zone 2 Connector
The ATCA specifications define five identical connectors for the Zone 2 data transport. These connectors are referred to as Free Board connectors and are assigned reference designators
P20 through P24 as per the ATCA specifications.
The ATCA-9405 is a node board and uses only two of the five connectors P20 and P23. P20 is used to support clock synchronization interface and update channels and P23 is used to support the base and fabric channels.
ATCA-9405 Installation and Use (6806800M71F) 43
Setup
44
Tyco HM-Zd Plus connectors are used to support 40 Gigabit Ethernet through backplane.
Table 2-12 Zone 2 Connector P20 Pin Assignment
Row Interface a
1 CLK CLK1A+
8
9
6
7
10
4
5
2
3
Update
Channel
Fabric
Channel 15
Fabric
Channel 14
Fabric
Channel 13 b
CLK1A-
NC
NC
NC
NC
NC
NC NC
UC_TX2+ UC_TX2-
UC_TX0+ UC_TX0-
NC NC
NC
NC
NC
NC
NC c
CLK1B+
NC
NC
NC
NC
NC
NC
UC_RX2+
UC_RX0+
NC d
CLK1B-
NC
NC
NC
NC
NC
NC
UC_RX2-
UC_RX0-
NC e
CLK2A+ f
CLK2Ag
CLK2B+ h
CLK2B-
NC
NC
NC
NC
NC
CLK3A+ CLK3ACLK3B+ CLK3B-
UC_TX3+ UC_TX3UC_RX3+ UC_RX3-
UC_TX1+ UC_TX1UC_RX1+ UC_RX1+
NC NC NC NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
Table 2-13 Zone 2 Connector P23 Pin Assignment
2
3
4
5
6
7
8
9
10
Row Interface a
1 Fabric
Channel 2
Fabric
Channel 1
NA
NA
NA
NA
Base
Channel 1
Base
Channel 2 b c d e
NC
NC
NC
NC
FC2_TX2+ FC2_TX2FC2_RX2+ FC2_RX2
-
FC2_TX0+ FC2_TX0FC2_RX0+ FC2_RX0
-
FC1_TX2+ FC1_TX2FC1_RX2+ FC1_RX2
-
FC1_TX0+ FC1_TX0FC1_RX0+ FC1_RX0
-
BC1_DA+ BC1_DABC1_DB+
BC2_DA+ BC2_DABC2_DB+
FC2_TX
3+
FC2_TX
1+
FC1_TX
3+
FC1_TX
1+
BC1_DBBC1_DC
+
BC2_DBBC2_DC
+
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC
NC f g h
FC2_TX3FC2_RX3+ FC2_RX3-
FC2_TX1FC2_RX1+ FC2_RX1-
FC1_TX3FC1_RX3+ FC1_RX3-
FC1_TX1FC1_RX1+ FC1_RX1-
BC1_DC-
BC2_DC-
NC
NC
NC
NC
BC1_DD+
BC2_DD+
NC
NC
NC
NC
BC1_DD-
BC2_DD-
NC
NC
NC
NC
ATCA-9405 Installation and Use (6806800M71F)
2.3.3.3.4 Zone 3 Connector
ATCA-9405 provides three connectors P30, P31, and P32 for connection to the RTM.
Table 2-14 Zone 3 Connector P30 Pin Assignments
Setup
Table 2-15 Zone 3 Connector P31 Pin Assignments
ATCA-9405 Installation and Use (6806800M71F) 45
Setup
Table 2-16 Zone 3 Connector P32 Pin Assignments
2.3.4
Debugging Headers
The following debugging headers are provided on the ATCA-9405. Please note that debugging headers are only used internally and may be removed later for cost reduction reasons. For more information regarding location and usage of debugging headers, please contact sales/marketing team.
46
Table 2-17 Debugging Header
Location Type
JP1 16-pin Header, 1.27mm pitch
Description
P2020 debugging header
MSRCID signals and second serial console port (UART1 interface)
JP2 8-pin Header, 1.27mm pitch P2020 Boot Flash header
JP3
JP5
P47
3-pin Header, 2.54mm pitch IPMC debug console
16-pin Header, 1.27mm pitch E10A-USB Emulator header for IPMC
P57
16-pin Header, 1.27mm pitch Packet Processor 1 debugging header
EJTAG support and second serial console port (UART1 interface)
16-pin Header, 1.27mm pitch Packet Processor 2 debugging header
EJTAG support and second serial console port (UART1 interface)
ATCA-9405 Installation and Use (6806800M71F)
Setup
Table 2-17 Debugging Header (continued)
Location Type
P7
Description
16-pin Header, 1.27mm pitch P2020 COP interface
P8 16-pin Header, 1.27mm pitch Management and Payload Power JTAG interface
2.3.4.1
IPMC Debug Console Header
A three pin header is provided to access IPMC debug console. The pin assignment is shown below.
Table 2-18 IPMC Debug Console Header Pin Assignment
Pin
1
3
4
Signal
GND
IPMC_TXD
IPMC_RXD
2.3.4.2
COP Header
A 16-pin header is provided for access to P2020 COP interface. The pin assignment is shown below.
Table 2-19 COP Header Pin Assignment
5
6
3
4
7
8
Pin Signal
1
2
TDO
QACK
TDI
TRST#
RUNSTOP
Target Power
TCK
CKSTP_IN#
ATCA-9405 Installation and Use (6806800M71F) 47
Setup
Table 2-19 COP Header Pin Assignment (continued)
14
15
16
Pin Signal
9 TMS
10
11
12
13
NC
SRESET#
GND
HRESET#
KEY
CKSTP_OUT#
GND
2.3.4.3
EJTAG Header
A separate 16-pin header is provided to access each Packet Processor EJTAG interface. The pin assignment is shown below.
Table 2-20 EJTAG Header Pin Assignment
7
8
9
10
11
12
5
6
3
4
Pin
1
2
Signal
TRST#
UART1_RXD
TDI
UART1_TXD
TDO
GND
TMS
GND
TCK
NC
RESET#
KEY
48 ATCA-9405 Installation and Use (6806800M71F)
Table 2-20 EJTAG Header Pin Assignment (continued)
Pin
13
14
15
16
Signal
NC
Target Power
NC
NC
The standard EJTAG connector has only 14 pins, thus do not connect to pin 15 and 16.
Setup
2.4
ATCA-9405 Setup
Following items are required to setup and check the operation of the ATCA-9405:
ATCA chassis and power supply
Console cables for EIA-232 ports (Cisco style pin out)
Computer terminal
Save the antistatic bag and box for future shipping or storage.
ATCA-9405 Installation and Use (6806800M71F) 49
Setup
2.4.1
Power Requirements
Make sure that the blade is used in an ATCA shelf connected to -60 VDC up to -48 VDC, according to Telecommunication Network Voltage (TNV-2). A TNV-2 circuit is a circuit whose normal operating voltages exceed the limits for a safety-extra-low-voltage (SELV) under normal operating conditions, and which is not subject to over-voltages from telecommunication networks.
Table 2-21 Typical Power Requirements
Configuration
Dual CN6880 running at 1.2GHz with 16GB DDR3 memory each,
P2020 running at 1.0GHz with 2GB DDR3 memory, 16GB USB Flash
Drive, Linux booted on P2020, Ethernet Switch fully initialized,
Cavium Octeon RiscCore Stress test running on all 64 CN6880 cores, board running at room temperature.
Dual CN6880 running at 1.0 GHz with 16GB DDR3 memory each,
P2020 running at 1.0GHz with 2GB DDR3 memory, 16GB USB Flash
Drive, Linux booted on P2020, Ethernet Switch fully initialized,
Cavium Octeon RiscCore Stress test running on all 64 CN6880 cores, board running at room temperature.
Dual CN6880 running at 800 MHz with 16GB DDR3 memory each,
P2020 running at 1.0GHz with 2GB DDR3 memory, 16GB USB Flash
Drive, Linux booted on P2020, Ethernet Switch fully initialized,
Cavium Octeon RiscCore Stress test running on all 64 CN6880 cores, board running at room temperature.
ARTM-9405B-16X10GE (maximum Power consumption)
Rated Voltage
Exception in the US and Canada
Operating Voltage
Exception in the US and Canada
Power
235 W
215 W
195 W
36 W
-48 VDC to -60 VDC
-48 VDC
-39 VDC to -72 VDC
-39 VDC to -60 VDC
The exact power requirements for the ATCA-9405 circuit board depends upon the specific configuration of the board, including the CPU frequency and amount of memory installed on the board.
50 ATCA-9405 Installation and Use (6806800M71F)
Setup
2.4.2
Environmental Considerations
As with any printed circuit board, make sure that the air flow to the board is adequate. Chassis constraints and other factors greatly affect the air flow rate. The environmental requirements are as follows:
Table 2-22 Environmental Requirements
Requirement
Temperature
Airflow
Operating
+5 ºC (+41 °F) to +40 ºC (+104 °F)
(normal operation) according to NEBS
Standard GR-63-CORE
-5 ºC (+23 °F) to +55 ºC (+131 °F)
(exceptional operation) according to
NEBS Standard GR-63-CORE
The blade is designed to operate in a chassis that provides 35 CFM across the blade for the stated temperature range
Temperature change
+/- 0.5 ºC/min according to NEBS
Standard GR-63-CORE
Relative humidity 5% to 90% non-condensing according to
Emerson-internal environmental requirements
Vibration 1 g from 5 to 100 Hz and back to 5 Hz at a rate of 0.1 octave/minute
Shock
Free fall
Half-sine, 11 m/Sec, 30 mSec/sec2
-
Non-Operating
-40 ºC (-40 °F) to +70 ºC (+158 °F)
(may be further limited by installed accessories)
+/- 0.5 ºC/min
5% to 95% non-condensing according to Emerson-internal environmental requirements
5-20 Hz at 0.1 g2/Hz
20-200 Hz at -3.0 dB/octave
Random 20-200 Hz at -3 m/Sec2
Blade level packaging
Half-sine, 6 mSec at 180 m/Sec2
1,200 mm/all edges and corners
1.0 m (packaged)
100 mm (unpacked)
ATCA-9405 Installation and Use (6806800M71F) 51
Setup
Figure 2-8 Air Flow Graph
52
During the safety qualification of this blade, the following on-board locations were identified as critical with regards to the maximum temperature during blade operation. To guarantee proper blade operation and to ensure safety, you have to make sure that the temperatures at the locations specified in the following table are not exceeded. If not stated otherwise, the temperatures should be measured by placing a sensor exactly at the given locations.
Table 2-23 Critical Temperature Limits
Component
Cavium CN6880 PP1
Cavium CN6880 PP2
DDR3 DIMM Modules
PCI-Express Switch
Ethernet Switch
Thermal Design Power
72 W
72 W
6.9 W
2.9 W
33.5 W
Maximum Case or Junction
Temperature
Tj = 101°C
Tj = 101°C
Tc = 85°C
Tj = 110°C
Tj = 115°C
If you integrate the blade in your own system, contact your local sales representative for further safety information.
ATCA-9405 Installation and Use (6806800M71F)
Setup
2.4.3
Hot Swap
The ATCA-9405 can be Hot Swapped, as defined in the ATCA specification. This section describes how to insert and extract an ATCA-9405 module in a typical ATCA system. (These procedures assume the system is using a shelf manager.)
The ATCA-9405 Rear Transition Module (RTM) has its own Hot Swap LED and switch, and it can be Hot Swapped in/out independently of the face plate. If the face plate is not present, then the RTM will not be powered. If the face plate is Hot Swapped out, the blue LED of the
RTM illuminates. In either case, the RTM can be safely removed.
Personal Injury or Product Damage
The product is supplied by a TNV-2 voltage. This voltage is considered hazardous. Make sure that the external power supply meets the relevant safety standards.
Make sure that TNV-2 is separated from dangerous voltages (mains) through double or reinforced insulation.
Inserting a board
1. Insert the ATCA-9405 into an available slot.
2. Push the face plate handle (tab).
The blue Hot Swap LED on the front panel (see
) flashes a long blink to indicate that the board insertion is in progress and the system management software is activating the slot.
Then the blue LED turns off, indicating the insertion process is complete, and payload power is present.
ATCA-9405 Installation and Use (6806800M71F) 53
Setup
Removing a board
1. Pull out the handle (tab) on the ATCA-9405 front panel at one click.
A short blink indicates that the board is requesting permission for extraction.
2. Remove the board when the blue LED on the front panel is on (no payload power).
Do not remove the ATCA-9405 while the blue LED is blinking.
2.5
Troubleshooting
In case of difficulty, use the following checklist:
Check that the ATCA-9405 circuit board is seated firmly in the carrier
Check that the system is not overheating
Check the cables and connectors to be certain that they are secure
Check that your terminal is connected to a console port
2.5.1
Technical Support
If you need help in resolving a problem with your ATCA-9405, visit www.emerson.com/embeddedcomputing . Keep the following information ready:
ATCA-9405 serial number identification, see Figure "Serial Number and Product ID" on page 27
.
Version and part number of the operating system (if applicable)
Whether your board has been customized for options such as a higher processor speed or additional memory
License agreements (if applicable)
54 ATCA-9405 Installation and Use (6806800M71F)
Setup
2.5.2
Product Repair
If you plan to return the board to Emerson Network Power for service, visit www.emerson.com/embeddedcomputing to obtain a Return Merchandise Authorization
(RMA) number. List the items that you are returning and the board serial number, plus your purchase order number and billing information if your ATCA-9405 hardware is out of warranty.
Contact our Test and Repair Services Department for any warranty questions. If you return the board, be sure to enclose it in an antistatic bag, such as the one in which it was originally shipped.
Put the RMA number on the outside of the package so that we can handle your problem efficiently. Our service department cannot accept material received without an RMA number.
ATCA-9405 Installation and Use (6806800M71F) 55
Setup
56 ATCA-9405 Installation and Use (6806800M71F)
Chapter 3
Packet Processor
3.1
Overview
Two independent Packet Processor Units are implemented on ATCA-9405. Each unit includes the Packet Processor with DDR3 memory, boot flash, Ethernet interfaces, USB interfaces, serial interfaces, power supply, and the connections to the blade infrastructure. The main functions
of the Packet Processor unit are illustrated in the Figure 3-1 on page 57
.
Figure 3-1 Packet Processor Unit Overview
ATCA-9405 Installation and Use (6806800M71F) 57
Packet Processor
The functions specified in the following sections are applicable for both the Packet Processor
Units.
3.2
CN6880 Processor
ATCA-9405 design is based on Cavium OCTEON II CN6880 Multi-Core MIPS64 Processors. The
CN68XX family is targeted for high-performance, high-throughput, service-rich applications in secure datacenter, mobile internet, and borderless enterprise applications.
The CN68XX family includes six software and pin-compatible processors, with 16 to 32 cnMIPS64 v2 cores, over 85 application acceleration engines, and real time Power Optimizer features.
High-bandwidth connectivity based on the latest standards-based SERDES I/Os including PCIe
Gen2, XAUI, DXAUI, RXAUI, SGMII, and Interlaken enables throughputs up to 40 Gbps using a single chip or scaling to over 100 Gbps using multiple chips.
Using up to four DDR3 controllers, a 4 MB L2 Cache, and complete application acceleration, including packet processing, Encryption/Decryption, Deep Packet Inspection (RegEx),
Compression/decompression, De-duplication, RAID, and Multi-core scaling, the CN68XX offers both the highest compute as well as the highest throughput processing and services.
Note that the 1.5 GHz version of the processor may not be used on ATCA-9405 because of thermal and electrical limitations. Tradeoff with regard to core count and memory size is needed to enable usage of 1.5 GHz processor.
3.3
Cache
The CN6880 includes 37 KB of L1 instruction cache and 32 KB of L1 data cache with parity protection and single-bit error correction for each core. The L1 caches are part of the processor core and run at full core clock frequency. Additionally, 4 MB L2 cache is shared between all processor cores. The L2 cache is 16-way set-associative with a 128 byte cache block, write-back and SECDED ECC support for both the on-chip data and tags.
58 ATCA-9405 Installation and Use (6806800M71F)
Packet Processor
3.4
System Memory
3.4.1
Memory Interface
All four CN6880 memory controllers are used on ATCA-9405. Each memory controller provides a 72-bit (64-bit data plus 8-bit for ECC) wide DDR3 interface channel that connects with a single
DDR3 DIMM socket.
Two physical memory banks (chip select signals) are implemented per DIMM socket to allow use of single-rank and dual-rank DIMM modules. The use of quad-rank DIMM modules is not supported by the memory controller.
Each DDR3 memory channel runs at data rates from 600 MHz to 1333 MHz providing a total bandwidth of up to 10.6 GBps per memory channel. The DDR3 speeds supported by CN6880 are listed in the table below.
Table 3-1 DDR3 Speed Grades
Standard
Name
DDR3-800
DDR3-1066
DDR3-1333
Memory
Clock
100 MHz
133 MHz
166 MHz
Cycle
Time
I/O Bus
Clock
10 ns 400 MHz
7.5 ns 533 MHz
6 ns 667 MHz
Data
Transfer Per
Second
800 Million
Module
Name
PC3-6400
Peak Transfer
Rate
6400 MBps
1066 Million PC3-8500 8533 MBps
1333 Million PC3-10600 10667 MBps
3.4.2
Memory Socket
Four VLP DDR3 DIMM sockets (one DIMM socket per memory channel) are provided to install in the shelf DIMM modules. The socket is keyed for DDR3 DIMM modules using 1.5V supply voltage.
ATCA-9405 Installation and Use (6806800M71F) 59
Packet Processor
3.4.3
Memory Modules
ATCA-9405 requires VLP DDR3 DIMM modules in order to fit within the maximum component height profile of an ATCA blade. Installed module height must not exceed 21.33 mm above PCB surface.
The CN6880 DIMM module requirements are listed below.
Table 3-2 CN6880 Memory Module Requirements
Characteristics
Form Factor
Memory Technology
DIMM Organization
Device Organization
Memory Size
Bus Width
Supply Voltage
Value
240 pin Very Low Profile (VLP) DIMM
DDR3-1333 (PC3-10600)
Registered DIMM
Single-rank or dual-rank x8 or x16 organized
2 GB, 4 GB, 8 GB
64-bit data
8-bit ECC
+1.5V
DIMMs with x4 memory devices are not supported by the memory controller.
The three preferred DIMM modules for CN6880 are listed in the table below, which provides up to 32 GB of DDR3 main memory per Packet Processor Unit.
Table 3-3 CN6880 Preferred Memory Modules
Characteristics
Vendor
Device
Size
Type
CAS latency
2 GB DIMM 4 GB DIMM 8 GB DIMM
Smart Modular Smart Modular Smart Modular
SG572568EMR069P2SG SG5721288EMR069P2SD SG5721G8EMR069P2SA
2 GB
DDR3-1333
9-9-9
4 GB
DDR3-1333
9-9-9
8 GB
DDR3-1333
9-9-9
60 ATCA-9405 Installation and Use (6806800M71F)
Packet Processor
Table 3-3 CN6880 Preferred Memory Modules (continued)
Characteristics
Organization
2 GB DIMM
18*128Mx8 (1Gb die)
4 GB DIMM
18*256Mx8 (2Gb die)
Number of Ranks 2
Package
2
240-pin VLP Registered DIMM with ECC
Oracle Number 9706802A83 9706802A73
8 GB DIMM
18*512Mx8 (4Gb die)
2
9706802A20
3.4.4
Thermal Sensor
Majority of DDR3 Registered DIMMs contain a Thermal Sensor on DIMM (TSOD) and its output is tied to EVENT# pin187 on the DIMM. This signal is an active low signal and is used by the
DIMM to notify the Processor or system management device that its thermal sensor has crossed a thermal threshold. The four thermal sensors are readable by CN6880 and by the
IPMC. The EVENT# interrupts are connected to CN6880 (which is responsible for interrupt routing) GPIO pins.
3.5
Octeon U-Boot
The two Octeon-II packet processors on the ATCA-9405 do not have a boot flash device. They are started using the oct-remote-boot command from the Octeon SDK via the PCI bus from the Freescale P2020 service processor.
Both Packet Processor Units are not powered when payload power is enabled by the IPMC. The
Packet Processor Units remain in power-off state until enabled by the Service Processor.
The CN6880 processors are hardware strapped for PCI Express (PCIe) Remote Boot Mode. After power up or reset the processor cores remain in idle state until they are booted via PCIe interface by the Service Processor.
3.5.1
NVRAM
U-Boot for the Octeon processors does not use an EEPROM to store its environment parameters. Instead, the configuration parameters for U-Boot are transferred together with the boot image from the service processor.
ATCA-9405 Installation and Use (6806800M71F) 61
Packet Processor
Application specific parameters can be kept on the service processor. It is not possible to save environment parameter changes within U-Boot as the saveenv command does not work.
3.5.2
Network Interfaces
The following network interfaces are available in U-Boot:
octmgmt0 (front-panel management interface) octeth0 (1000BaseX interface to switch)
octeth4 (DXAUI interface 0 to switch) octeth5 (DXAUI interface 1 to switch)
3.6
SerDes Configuration
Packets and control information can flow from the CN6880 via any of the SGMII, XAUI, DXAUI,
RXAUI, Interlaken, or PCIe interfaces. Internally CN6880 supports five SerDes quad-lane modules (QLMs) for a total of 20 SERDES lanes.
For the default configuration of ATCA-9405 (where the Packet Processors are interconnected via Interlaken Interface), one PCI Express, one Interlaken, two DXAUI and two RXAUI interfaces are provided.
Note that ATCA-9405-ILK-MODULE must be assembled on this version.
The corresponding QLM mode strapping is listed in the table below. The QLM reference clock must be configured according to the IO interface standard used.
Table 3-4 CN6880 SerDes Configuration (Interlaken Mode)
QLM
QLM0
QLM1
QLM2
Supported Interfaces
XAUI, RXAUI, SGMII
PCI-Express, Interlaken
PCI-Express, Interlaken XAUI,
DXAUI, SGMII
Configuration
RXAUI
156.25 MHz
Interlaken
156.25 MHz
DXAUI
156.25 MHz
62 ATCA-9405 Installation and Use (6806800M71F)
Packet Processor
Table 3-4 CN6880 SerDes Configuration (Interlaken Mode) (continued)
QLM
QLM3
QLM4
Supported Interfaces
PCI-Express XAUI, SGMII
PCI Express XAUI, DXAUI, SGMII
Configuration
PCI Express Gen 1
100 MHz
DXAUI
156.25 MHz
3.7
PCI Express Interface
CN6880 integrates two PCIe Interfaces that are compliant with the PCIe Base Specification
Revision 2.0. They are configured at boot time to act as either root complex or endpoint. The
PHY of the PCIe interface supports transmission rate of up to 5.0 Gbps per lane. The two interfaces can be configured as one single or two independent ports with x1, x2, x4, or x8 link widths.
On ATCA-9405, PCIe Interface 0 (using QLM3 SerDes module) is configured to x1 link width and is connected to the PLX PEX8608 switch. For more details regarding PCIe infrastructure, refer
to Service Infrastructure on page 99
.
PCIe Interface 1 is not used on ATCA-9405.
3.8
Ethernet Interface
The CN6880 provides a number of integrated on-chip Ethernet controllers and supports various interfaces standards. The CN6880 family can implement up to:
16 SGMII/1000BASE-X SerDes interfaces through up to 4 four-port (four-lane) packet interfaces. A full-duplex port consists of four external pins, a differential output pair and a differential input pair. CN6880 couples logic that implements the SGMII and/or 1000BASE-
X protocols on SerDes lanes with a 10/100/1000 802.3 MAC.
Five XAUI SerDes interfaces in up to 4 four-lane packet interfaces. Each interface consists of 16 external pins in total, four differential output pairs plus four differential input pairs.
CN6880 couples logic that implements the XAUI, reduced XAUI (RXAUI) or double data rate XAUI (DXAUI) interface/protocols on SerDes lanes with an IEEE 802.3-2005 MAC.
ATCA-9405 Installation and Use (6806800M71F) 63
Packet Processor
In addition, CN6880 provides a 10/100/1000 Ethernet MAC-to-PHY interface supporting either an MII MAC or an RGMIII MAC. The MII links are fully compliant with the IEEE 802.3 specifications, and the RGMII links are fully compliant with the HP RGMII 1.3 specifications.
On ATCA-9405 two DXAUI, two RXAUI, and one RGMII interface are implemented.
3.8.1
Front Panel Interface
The RGMII interface is connected to a 10/100/1000Base-T port at the front panel. The physical interface is implemented using Marvell 88E1512 Single Port Gigabit Ethernet Transceiver and an RJ45 connector with integrated magnetics.
3.8.2
Base and Fabric Interface
The SGMII interface (using QLM0 SerDes module) is connected to the Ethernet Switch and provides a Gigabit Ethernet interface to the control plane (Base Interface).
The two DXAUI interfaces (using QLM2 and QLM4 SerDes module) are connected to the
Ethernet Switch and provide a 2x20G Ethernet interface to the data plane (Fabric Interface).
The two RXAUI interfaces (using QLM0 SerDes module) are connected to the Ethernet Switch and provide a 20 Gigabit Ethernet interface to the data plane (FI).
3.9
Interlaken Interface
The Interlaken interface unit provides a narrow, high-speed, channelized packet interface using the scalable Interlaken protocol. It conforms to the Interlaken Protocol Definition V1.2 and the
Interlaken Look-Aside Protocol Definition V1.1. Interlaken combines the channelization, programmable burst sizes, and per-channel backpressure features of the SPI4.2 protocol with the long-reach and reduced pin count of the XAUI protocol, and it offers the ability to tailor the design to the interface capacity of the application.
The Interlaken interface unit supports SerDes lanes running at up to 6.25 Gbps, two links/interfaces, and x8, x4, x2, and x1 configurations. On ATCA-9405 the Interlaken interface
(using QLM1 SerDes module) is used for interconnection between the two Packet Processors
(when ATCA-9405-ILK-MODULE is assembled).
64 ATCA-9405 Installation and Use (6806800M71F)
Packet Processor
3.10 USB Interface
The CN6880 USB subsystem contains an EHCI host controller, an OHCI host controller, and two physical USB ports. The USB interface is with USB specification, Revision 1.1 and supports highspeed (480 Mbps), full-speed (12 Mbps), and low-speed (1.5 Mbps) operation. For more information refer to the Cavium OCTEON II Hardware Reference Manual.
The USB interface 1 is not used on ATCA-9405.
An assembly option (footprint for a high speed USB switch type FSUSB31K8X) to allow connection of USB Interface 0 toward ARTM is available. If you require this, then contact the sales/marketing team.
3.11 UART Interface
The CN6880 provides a dual UART compatible with the PC16550D. Both interfaces consist of four wires RXD, TXD, RTS, and CTS. The interfaces run in full duplex mode and the baud rates are software programmable.
The first interface COM0 is connected to the Glue Logic FPGA that routes the interface to either the front panel connector or to the Terminal server. The connection to the front panel console connector is implemented via RS-232 transceiver. All four signals (RXD, TXD, RTS, and CTS) are connected to an RJ45 connector using Cisco DTE pin out assignment. Serial port parameters for the front panel interface are 9600 baud, 8 data bits, no parity bit, and 1 stop bit.
When serial port COM0 is routed to the Terminal Server for console redirection into the base network, only TXD and RXD signals are used. IPMC can control serial port routing. Serial port parameters for this interface are 9600 baud, 8 data bits, no parity bit, and 1 stop bit. For more information, see
Serial Line Selection on page 137 .
The second interface COM1 (only signals TXD and RXD) is connected to a standard 16-pin header for debugging purposes.
ATCA-9405 Installation and Use (6806800M71F) 65
Packet Processor
3.12 I2C Interface
The CN6880 has 2 two-wire serial interfaces (TWSI) that provide multi-master and masterslave I2C mode support. TWSI interface 0 is connected to the four DIMM modules for SPD detection. TWSI interface 1 is connected to the external EEPROM and to the Glue Logic FPGA for access to internal status and control registers.
Table 3-5 CN6880 I2C Bus Assignment
Address
0xA0
0xA2
0xA4
0xA6
0xA0
0xFE
1
1
0
0
TWSI
0
0
Component Function
NA
NA
DIMM1 Memory Module SPD PROM
DIMM2 Memory Module SPD PROM
NA
NA
24LC128
NA
DIMM3 Memory Module SPD PROM
DIMM4 Memory Module SPD PROM
U-Boot Parameter Storage EEPROM
Glue Logic FPGA
3.13 JTAG Interface
The IEEE 1149.1 compliant JTAG boundary scan interface of CN6880 is connected to an onboard 16-pin header to support a processor emulator for board debug. The additional signals for MIPS EJTAG support according to EJTAG Specification Revision 5.0 are supported.
For connector pin out details, refer to
.
3.14 Interrupts
Each Packet Processor is responsible for handling of interrupts that are generated within its own domain.
3.14.1 Packet Processor Interrupts
The CN6880 includes a central interrupt unit that allows centralized collection of internal and external interrupts as well as interrupt selection and distribution to the cores and to the integrated PCI Express controllers.
66 ATCA-9405 Installation and Use (6806800M71F)
Packet Processor
Up to 16 GPIO pins are provided by the CN6880 to connect external interrupts with the central interrupt unit. Each GPIO pin can be programmed to be a level-sensitive interrupt pin or an edge-triggered interrupt pin.
All interrupt sources from the packet processor domains are directly connected to the
GPIO[5:1] inputs of the associated Packet Processor.
The following table lists all interrupt sources of the packet processor domains and how the interrupts are routed on the board:
Table 3-6 Packet Processor Interrupts
Interrupt Source
PP1
PP2
Interrupt Name
PP1_DDR0_EVENT_N
PP1_DDR1_EVENT_N
PP1_DDR2_EVENT_N
PP1_DDR3_EVENT_N
PP1_ETH_INT_N
PP2_DDR0_EVENT_N
PP2_DDR1_EVENT_N
PP2_DDR2_EVENT_N
PP2_DDR3_EVENT_N
PP2_ETH_INT_N
Usage
DIMM1 Thermal Event
DIMM2 Thermal Event
DIMM3 Thermal Event
DIMM4 Thermal Event
88E1512 PHY Interrupt
DIMM1 Thermal Event
DIMM2 Thermal Event
DIMM3 Thermal Event
DIMM4 Thermal Event
88E1512 PHY Interrupt
IRQ Line
PP1_GPIO2
PP1_GPIO3
PP1_GPIO4
PP1_GPIO5
PP1_GPIO1
PP2_GPIO2
PP2_GPIO3
PP2_GPIO4
PP2_GPIO5
PP2_GPIO1
3.15 Power Supply
ATCA-9405 supports software controlled power-down and power-on sequence for the two
Packet Processor units as described in Hot Plug Support on page 100 .
ATCA-9405 Installation and Use (6806800M71F) 67
Packet Processor
3.16 Cooling
A passive heat sink is mounted on top of the CN6880 package. The heat sink is designed to withstand shock and vibration tests according to the environmental conditions. The heat sink keeps the processor die temperature below the maximum rating of 125°C under any conditions listed in
Environmental Considerations on page 51 .
68 ATCA-9405 Installation and Use (6806800M71F)
Chapter 4
Service Processor
4.1
Overview
The Service Processor (SP) functional unit is in charge of blade and RTM infrastructure. This includes management and control of the two CN6880 Packet Processors, the 98CX8234
Ethernet Switch, the PEX8608 PCI e Switch, and the RTM. The SP also takes the root complex role for the complete PCIe infrastructure on the blade and RTM. The main functions of the SP are shown in the
Figure 4-1 Service Processor Unit Overview
ATCA-9405 Installation and Use (6806800M71F) 69
Service Processor
4.2
P2020 Processor
A Freescale P2020 QorIQ Communication Processor is used on ATCA-9405 as the onboard SP.
The processor is manufactured in 45nm process technology and combines dual Power
Architecture™ e500v2 processor cores with system logic required for networking, wireless infrastructure, and telecommunications applications.
The P2020 is available with three speed grades running at 800 MHz, 1000 MHz, and 1200 MHz core clock frequency. All speed grades are supported, the default speed grade on ATCA-9405 is 1.0GHz. Additionally, the board is prepared to support all three speed grades of P2010 single core derivative.
4.3
Cache
The P2020 includes 32 KB of L1 instruction cache and 32 KB of L1 data cache with parity protection for each core. L1 caches can be locked entirely or on a per-line basis, with separate locking for instructions and data. The 512 KB L2 cache is common to both processors and has full ECC protection on 64-bit boundary and supports instruction caching, data caching, or both modes. The L1 and L2 caches are part of the P2020 core complex and run at full core clock frequency.
4.4
Main Memory
4.4.1
Memory Interface
The P2020 memory controller on ATCA-9405 is configured for DDR3 SDRAM mode. The controller provides a 72-bit (64-bit data plus 8-bit for ECC) wide DDR3 interface (channel) that connects the P2020 with a single DDR3 DIMM socket.
Two physical memory banks (chip select signals) are implemented on DIMM socket to allow use of single-rank and dual-rank DIMM modules.
70 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
The DDR3 memory interface runs at a maximum frequency of 400 MHz providing a total bandwidth of 6.4 GBps. The fastest supported DDR3 speed is DDR3-800 (PC3-6400).
Table 4-1 P2020 DDR3 Speed Grades
Standard
Name
DDR3-800
Memory
Clock Cycle Time
100 MHz 10 ns
DDR3-1066 133 MHz 7.5 ns
DDR3-1333 166 MHz 6 ns
I/O Bus
Clock
Data Transfers per second
400 MHz 800 Million
533 MHz 1066 Million
667 MHz 133 Million
Module
Name
PC3-6400
Peak Transfer
Rates
6400 MBps
PC3-8500 8533 MBps
PC3-10600 10667 MBps
4.4.2
Memory Socket
A Very Low Profile (VLP) DDR3 DIMM socket is provided to install of-the-shelf DIMM modules.
The socket is keyed for DDR3 DIMM modules using 1.5V supply voltage.
4.4.3
Memory Modules
ATCA-9405 requires VLP DDR3 DIMM modules in order to fit within the maximum component height profile of an ATCA blade. Installed module height must not exceed 21.33 mm above PCB surface.
Table 4-2 P2020 Memory Module Requirements
Characteristics
Form Factor
Memory Technology
DIMM Organization
Device Organization
Memory Size
Bus Width
Supply Voltage
Value
240 pin Very Low Profile (VLP) DIMM
DDR3-800 (PC3-6400)
Registered DIMM
Single-rank or dual-rank x8 or x16 organized
2 GB, 4 GB, 8 GB
64-bit data
8-bit ECC
+1.5V
ATCA-9405 Installation and Use (6806800M71F) 71
Service Processor
DIMMs with x4 memory devices are not supported by the memory controller.
Default main memory size is 2 GB using one Smart Modular SG572568EMR069P2SG DIMM module. The three preferred DIMM modules for P2020 are listed in the table below, providing up to 8 GB of DDR3 main memory.
Table 4-3 P2020 Preferred Memory Modules
Characteristics
Vendor
Device
Size
2 GB DIMM
Smart Modular
4 GB DIMM
Smart Modular
8 GB DIMM
Smart Modular
SG572568EMR069P2SG SG5721288EMR069P2SD SG5721G8EMR069P2SA
2 GB 4 GB 8 GB
Type
CAS latency
DDR3-1333
9-9-9
Organization 18*128Mx8 (1Gb die)
Number of Ranks 2
Package
Oracle Number
DDR3-1333
9-9-9
18*256Mx8 (2Gb die)
2
240-pin VLP Registered DIMM with ECC
9706802A83 9706802A73
DDR3-1333
9-9-9
18*512Mx8 (4Gb die)
2
9706802A20
4.4.4
Persistent Memory
The P2020 offers both hardware and software options to support persistent main memory.
The persistent memory is an array of random access memory that preserves its contents during a warm/soft reset. Persistent memory is a necessary prerequisite for performing post mortem analysis of log data after reset and reboot of the payload CPU.
For persistent memory support on ATCA-9405, a dedicated register is implemented in the Glue
Logic FPGA to enable or disable persistent memory by software. If persistent memory is enabled, the contents of the main memory stay unchanged after any applied reset, except power-up reset.
After reset is released and the board is booted, the application must check the persistent memory bit in FPGA and (if set) initialize the main memory but keep persistent memory area untouched.
72 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
4.4.5
Thermal Sensor
Majority of DDR3 Registered DIMMs contain a Thermal Sensor on DIMM (TSOD) and its output is tied to EVENT# pin187 on the DIMM. This signal is an active low signal and is used by the
DIMM to notify the Processor or system management device that its thermal sensor has crossed a thermal threshold. The thermal sensor is readable by P2020 and by the IPMC. The
EVENT# interrupt is connected to the FPGA (which is responsible for interrupt routing).
4.5
SP U-Boot
After power up or reset, the following necessary steps are taken by P2020 processor:
boot from active SPI Boot Flash
initialize PCIe infrastructure initialize network infrastructure
power-up Packet Processors initialize PCIe interfaces of Packet Processors boot Packet Processors via PCIe
4.5.1
Environment Variables
U-Boot uses environment variables to both control the behaviour of various U-Boot components and to report information to user applications.
The working set of the U-Boot environment variables is stored in memory and can be accessed using the setenv and getenv commands. The values of environment variables can be:
The content of the NVRAM (an I2C EEPROM), or (if the NVRAMs CRC is invalid) a set of compiled-in defaults.
The content of the IPMI boot parameter storage
Dynamic variables set during the boot-up phase
Variables set by the user via the U-Boot shell.
ATCA-9405 Installation and Use (6806800M71F) 73
Service Processor
The above listing basically describes the order of precedence, that is a variable stored in the
NVRAM can potentially be overridden, modified, or deleted by a variable of the same name stored in the IPMI boot parameter storage, and later on by dynamic variables declared during the boot process or by the user from the shell prompt.
Although the saveenv command stores the complete set of variables currently being used into the NVRAM, the actual setting of these variables might not become valid because they are overridden by the other entities.
Note that the U-Boot does not synchronize the NVRAM storage and the IPMI boot parameter storage by itself.
If the NVRAMs CRC is invalid, the NVRAM is initialized with the blade's default values immediately after detecting the bad CRC (before reading additional values from the IPMC).
4.5.2
Passing Parameter Set to the Operating System
An operating system making use of environment variables should only rely on the RAM copy of the parameter set.
U-Boot passes its current parameters via the firmware device tree. It generates a node /uboot-env
, containing a node for each environment variable. This is done by the bootm command as the final step before executing the OS image. Under Linux, these parameters appear under the directory /proc/dev-tree/u-boot-env.
4.5.3
Dynamic Variables Set During the Boot Phase
This section lists the variables that are set during the boot phase.
Table 4-4 Dynamic Variables Set During the Boot Phase
Variable Name boottime physical_slot logical_slot bootbank
Description
The time (UNIX time) when the blade has started. If this variable is set to "-1" then the RTC does not run at startup
The physical slot number where the blade is mounted
The logical slot number where the blade is mounted
The physical boot bank number from which the blade has started (0 or
1)
74 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
Table 4-4 Dynamic Variables Set During the Boot Phase (continued)
Variable Name shelf_id shelf_location dhcp_clid uboot_version ethaddr eth1addr eth2addr fpga_version
Description
The first byte of the shelf identifier as read from the shelf manager
Two bytes encoding the logical slot number and the 1st byte of the shelf
ID, each octet separated by a colon
The DHCP client ID (option 61) used in DHCP discover
The U-Boot firmware version (in the format V<x>.<y>.<z>).
The Ethernet MAC addresses for the local network interfaces
Version of Glue Logic FPGA
4.5.4
Variables for Controlling the Boot Progress
Table 4-5 Variables for Controlling the Boot Progress
Variable Name bmc_wd_timeout ethdefault bootcmd bootargs bootdelay serverip
Default undefined undefined run $ramboot undefined
10 undefined
Description
Configures the IPMI watchdog time-out for booting the OS in seconds. Setting it to -1 (default) disables the IPMI watchdog.
The default network interface to be configured. If set, the ethact variable is initially assiged to the value of this parameter.
A sequence of commands executed to boot the operating system (via auto boot or the boot command)
Arguments for the operating system (that is, the linux kernel command line or the vxWorks boot parameters).
Delay in seconds before the automatic boot sequence starts.
A value of "-1" disables autoboot
The IP address of the boot server
ATCA-9405 Installation and Use (6806800M71F) 75
Service Processor
Table 4-5 Variables for Controlling the Boot Progress (continued)
Variable Name ipaddr gatewayip pram
Default undefined undefined
0
Description
The IP address used for network communication
The default gateway IP
This is a decimal value encoding a memory size in kilobytes (1024 bytes). The specified amount of memory is reserved at the end of physical memory and will not be overwritten by U-Boot. This memory will also be retained over reset.
The size of usable memory is reported in the variable "pram_size".
4.5.5
Firmware Update
The command firmware is used to show the versions of various components and to upgrade some of them.
The show command displays version information
The update command requires a valid firmware image in memory. It supports the following component:
– U-Boot for P2020
The tool always programs the standby image and after successful update, activates it to be used at the next reboot.
It also activates the failsafe logic. It is in the responsibility of the end-user to deactivate the fail-safe logic once the boot sequence is successful. The fail-safe logic can be deactivated via IPMI using the OEM command set feature configuration as described in
Set Feature Configuration on page 149 .
Programming a firmware image requires that the image is present as .fri file. The firmware update command will verify that the contents of the fri-file are suitable to be programmed into the selected device.
4.5.6
Application/OS Boot
The default boot device of the ATCA-9405 is the internal USB disk.
76 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
4.5.6.1
Default Boot Sequences
Various default boot sequences are stored in environment variables. These variables can be executed as script using the run command (for example, run nfsboot).
4.5.6.1.1 Disk (bootcmd)
This is the default boot operation (bootcmd). The boot command is run usbboot
4.5.6.1.2 TFTP/NFS (nfsboot)
Load the kernel $bootfile via TFTP from the nfs server $serverip, path $rootpath, and boot it using $serverip$rootpath as nfsroot, and $netdev as linux network device.
The boot command is run nfsboot
Netdev value is eth0 for front panel connector MGT ETH
4.5.7
Memory/Address Map Initialization
4.5.7.1
Address Map
U-Boot configures the P2020 address map via local access windows (LAWs) and MMU mapping entries (TLBs) as described in the following table:
Table 4-6 Address Map
Device
DDR3
PCI1 Memory
PCI1 I/O
FPGA
Physical Address Range Virtual Start Address
0x0.0000.0000
0x0.xxxx.xxxx
0x0000.0000
0xc.0000.0000
0xf.ffc0.0000
0xf.ef00.0000
0x8000.0000
0xffc0.0000
0xef00.0000
P2020 CCSR 0xf.ffe0.0000
0xffe0.0000
Note
2GB or 4GB DDR3 memory
1GB Non-prefetchable
256KB I/O space
128 bytes FPGA register set
P2020 CCSR space
ATCA-9405 Installation and Use (6806800M71F) 77
Service Processor
4.6
Local Bus
The P2020 includes an Enhanced Local Bus Controller (eLBC). The main component of the eLBC is its memory controller, which provides a seamless 16-bit interface to many types of memory devices and peripherals. The memory controller is responsible for controlling eight memory banks (chip selects) shared by a general-purpose chip select machine (GPCM), an FCM, and up to three UPM machines. The eLBC offers a multiplexed 16-bit address and data bus operating at up to 83 MHz. Data checking and protection features, such as parity support and write protection are included.
The eLBC supports ratios of 4, 8, and 16 between the faster internal CCB clock and slower external bus clock. The selected divider for ATCA-9405 is 16, resulting in a local bus frequency of 31 MHz at a platform frequency of 500 MHz (1000 MHz processor).
The eLBC provides one GPCM, one FCM, and three UPMs for the local bus to allow the implementation of memory systems with very specific timing requirements. The GPCM provides interfacing for simpler, lower-performance memories, and memory-mapped devices.
It has inherently lower performance because it does not support bursting. For this reason,
GPCM-controlled banks are used primarily for boot-loading from NVRAM or NOR Flash, and access to low-performance memory-mapped peripherals.
On ATCA-9405, the GPCM provides the local bus interface to the FPGA. The FCM and UPM controllers are not used. The FPGA is connected to chip select LCS3# and configured as 8-bit device.
Contact sales/marketing team, if you require a detailed CPLD and FPGA specification.
4.7
SerDes Configuration
The P2020 provides four SerDes blocks that support the SGMII, serial RapidIO, and PCIe highspeed I/O interface standards. Each SerDes can be mapped to one of the different IO interfaces depending on power-on reset (POR) configuration.
78 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
Two SGMII and one PCIe interface are needed for ATCA-9405. The corresponding SerDes configuration is listed in the table below. The SerDes reference clock for this configuration must be 100 MHz.
Table 4-7 P2020 SerDes Configuration
Reset Configuration cfg_IO_ports[0:3]
Value
1110 cfg_sgmii2 cfg_sgmii3
0
0
SerDes Configuration
SerDes Lane 0: PCI Express 1 (x1, 2.5 Gbps)
SerDes Lane 1: PCI Express 2 (x1, 2.5 Gbps) - not used
SerDes Lane 2: SGMII eTSEC2 (x1, 1.25 Gbps)
SerDes Lane 3: SGMII eTSEC3 (x1, 1.25 Gbps) eTSEC2 operates in SGMII mode eTSEC3 operates in SGMII mode
SerDes Lane 1 is not used on ATCA-9405, so it is left unconnected.
4.8
PCI Express Interface
The P2020 supports three PCIe interfaces that are compliant with the PCIe Base Specification
Revision 1.0a. They are configued at boot time to act as either root complex or endpoint. The
PHY of the PCIe interface operates at a transmission rate of 2.5 Gbps (data rate of 2 Gbps) per lane. The ports can be configured for x1, x2, or x4 link widths.
On ATCA-9405, PCIe Port 1 (SerDes Lane 0) is configured to x1 link width and is connected to the PLX PEX8608 switch. P2020 acts as root complex and is responsible for blade control and monitoring as well as PCIe hot plug handling. The remaining two PCIe Ports are not used.
For more details regarding PCIe infrastructure and hot plug handling, refer to
.
ATCA-9405 Installation and Use (6806800M71F) 79
Service Processor
4.9
Ethernet Interface
The P2020 has three on-chip enhanced three-speed Ethernet controllers (eTSECs). The eTSECs incorporate a media access control (MAC) sub layer that supports 10/100/1000 Mbps
Ethernet/802.3 networks with SGMII, GMII, RGMII, MII, RMII, TBI, and RTBI physical interfaces, as well as 8-bit or 16-bit FIFO interfaces that bypass the Ethernet MAC.
The eTSECs include 2 KB receive and 10 KB transmit FIFOs and DMA functions, support for programmable CRC generation and checking, RMON statistics, jumbo frames, TCP/IP acceleration and QoS features, VLAN insertion and deletion and MAC address recognition.
On ATCA-9405, eTSEC1 is configured to RGMII mode while eTSEC2 and eTSEC3 are configured to SGMII mode. The corresponding strapping is listed in the table below.
Table 4-8 P2020 eTSEC Configuration
Reset Configuration cfg_tsec_reduce cfg_tsec1_prtcl[0:1] cfg_sgmii2 cfg_sgmii3
0
0
Value
0
10 eTSEC Configuration eTSEC1 operates in RGMII mode eTSEC2 operates in SGMII mode eTSEC3 operates in SGMII mode
4.9.1
Front Panel Interface
P2020 eTSEC1 interface is connected to a 10/100/1000Base-T port at the front panel. The physical interface is implemented using Marvell 88E1512 Single Port Gigabit Ethernet
Transceiver (connected to the P2020 in RGMII mode) and an RJ45 connector with integrated magnetics.
4.9.2
Base and Fabric Interface
The eTSEC2 and TSEC3 interfaces are connected to the Ethernet Switch and provide a Gigabit
Ethernet interface to both the control plane (base interface) and the data plane (fabric interface). The two eTSECs are configured for SGMII mode and are directly connected to the
Ethernet Switch.
80 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
4.10 SPI Interface
The P2020 includes a full duplex four-wire SPI interface. The SPI interface can support up to four separate SPI devices. The P2020 SPI interface connects to the SPI Boot Flashes, to the Telecom
DPLL and to the Terminal Server. The following table shows the P2020 chip select assignments for the SPI interface.
Table 4-9 P2020 SPI Chip Select Assignment
SPI Chip Select
CS0#
CS1#
CS2#
CS3#
Value
SPI Boot Flash 1 (default)
SPI Boot Flash 2 (backup)
Telecom DPLL
Terminal Server
4.10.1 Boot Flash
Two SPI Flash devices are provided as P2020 boot devices.
Hardware write protection signal WP# of SPI Flash is pulled down permanently to enable the hardware protection built in the SPI Flash. Individual Boot Flash sector write protection must be implemented via software mechanism according to SPI Flash specification.
4.10.2 Boot Flash Selection
For crisis recovery, Boot Flash 1 (default) and Boot Flash 2 (recovery) can be exchanged under
IPMC software control through logic implemented in the FPGA.
ATCA-9405 Installation and Use (6806800M71F) 81
Service Processor
By default, the P2020 boots from Boot Flash 1. An IPMI OEM command can be used to send a message to the IPMC to change the boot device. The IPMC provides an IPMI sensor to control the signal BOOT_SELECT. If the BOOT_SELECT signal is set high, the payload processor boots from Boot Flash 2 after reset.
Table 4-10 P2020 Boot Flash Selection
OFF
OFF
ON
SW1-3
Boot from debug header
OFF
OFF
-
ON
ON
SW1-1
IPMC or manual boot selection
OFF
OFF
-
ON
OFF
-
-
SW1-2
Manual SPI
Flash selection
-
-
-
IPMC
BOOT_
SELECT
LOW
HIGH
SPI Flash 0
(Default)
Selected
Selected
SPI Flash 1
(Recovery)
Selected
Selected
Debug
Header
Selected
4.11 USB Interface
The P2020 provides an USB 2.0 compliant controller which can be configured to operate as a stand-alone host or stand-alone device. The controller is enhanced host controller interface
(EHCI) compatible and supports high-speed (480 Mbps), full-speed (12 Mbps), and low-speed
(1.5 Mbps) operation. The USB interface requires an external PHY with a UTMI+ low pin count interface.
The USB PHY is configured as host controller and connects to an USB hub in order to provide multiple USB ports. A high-speed USB 2.0 hub controller is used. The device offers two downstream ports supporting high-speed, full-speed, and low-speed operation.
The hub provides fully integrated USB termination and pull-up/pull-down resistors, over current protection and power supply for the USB ports. USB port 1 is connected to an USB connector at the front panel. USB port 2 is connected to an on-board eUSB Flash Drive.
4.11.1 USB Connector
One standard USB type A connector is provided at the face plate for USB access to the SP. For connector pin out details, refer to
82 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
4.11.2 e-USB Flash Drive
ATCA-9405 provides a factory option for eUSB Flash Drives. The Flash Drive can be used for local mass storage, persistent user data storage or booting of operating system and user images. A low profile 2mm eUSB header on the board is connected to the P2020 to support eUSB modules from various vendors.
The default assembly SG9ED52M1GG4NEMR from Smart Modular Technology provides 16 GB of User Flash memory. The module provides sustained read speeds of up to 35 MBps and writes
.
4.12 UART Interface
The P2020 provides a dual UART compatible with 16450 and the PC16550D. Both interfaces consist of four wires RXD, TXD, RTS, and CTS. The interfaces run in full duplex mode and the baud rates are software programmable.
The first interface COM0 is connected to the FPGA that is able to route the interface to either the front panel connector and/or to the Terminal server. The connection to the front panel console connector is implemented via RS-232 transceiver. All four signals (RXD, TXD, RTS, and
CTS) are connected to an RJ45 connector. For connector pin out, refer to Serial Console Ports.
Serial port parameters for the front panel interface are 9600 baud, 8 data bits, no parity bit, and
1 stop bit.
When serial port COM0 is routed to the Terminal Server for console redirection into the base network, only TXD and RXD signals are used. Serial port parameters for this interface are 9600 baud, 8 data bits, no parity bit, and 1 stop bit.
The second interface COM1 (only signals TXD and RXD) is connected to a standard 16-pin header for debugging purposes.
4.13 I2C Interface
The P2020 includes a dual I2C controller with open-drain two-wire interfaces that provides multi-master and master-slave I2C mode support.
ATCA-9405 Installation and Use (6806800M71F) 83
Service Processor
I2C bus 1 is used to connect to a boot sequencer memory and the SPD PROM device of the
DDR3 memory module. The boot sequencer can be optionally used for initialization of the
P2020.
I2C bus 2 is connected to an onboard RTC (DS1337).
Table 4-11 P2020 I2C Bus Assignment
Address
0xA0
0xA2
0xD0
1
2
Bus
1
Component
AT24C64C
SPD
DS1337U+
Function
Boot Sequencer ROM
DIMM Module SPD PROM
Real Time Clock
4.13.1 Real Time Clock (RTC)
The blade provides an I2C™-bus-compatible real-time clock type DS1337 from Maxim. This device contains a real-time clock/calendar and 31 bytes of static random access memory
(SRAM). The real-time clock/calendar provides seconds, minutes, hours, day, date, month, and year information. The end of the month date is automatically adjusted for months with fewer than 31 days, including corrections for leap year up to the year 2100. The clock operates in either the 24hr or 12hr format with an AM/PM indicator. A battery socket is provided on the board to backup the RTC power supply.
4.14 JTAG Interface
The IEEE 1149.1 compliant JTAG boundary scan interface of the P2020 is connected to an onboard 16-pin COP header to support a processor emulator for board debug. For connector
pin out details, refer to Connector Pin Assignment .
4.15 Interrupts
The Service Processor is responsible for handling of infrastructure related interrupts.
84 ATCA-9405 Installation and Use (6806800M71F)
Service Processor
4.15.1 Service Processor Interrupts
The P2020 provides a programmable interrupt controller with up to twelve interrupt request inputs with programmable polarity and sense of each signal. Only interrupt inputs IRQ[6:0] are used on ATCA-9405.
The following table lists all interrupt sources of the Service Processor domain and how the interrupts are routed on the board.
Table 4-12 Service Processor Interrupts
Interrupt Source
P2020
Interrupt Name
SP_INT_OUT#
PEX8608
88X2241
ACS8525
98CX8234
88E1322
88E1512
SP_DIMM_EVENT_N
SP_RTC_IRQ_N
PEX_INTA_N
PEX_FATA_ERR_N
FAB_CH1_LASI[3:0]_N
FAB_CH2_LASI[3:0]_N
LCCB_INTREQ_N
SW_INT[3:0]_N
BASE_PHY_INT_N
SP_ETH_INT_N
Comments
Persistent Memory
Interrupt (IRQ_OUT#)
DIMM Thermal Event
IRQ Line
IRQ[6:0]
IRQ[6:0]
Real Time Clock Interrupt IRQ[6:0]
IRQ[6:0]
IRQ[6:0]
IRQ[6:0] Link Alarm Status for
Fabric Channel 1
Link Alarm Status for
Fabric Channel 2
IRQ[6:0]
IRQ[6:0]
IRQ[6:0]
IRQ[6:0]
IRQ[6:0]
4.16 Cooling
A passive heat sink is mounted on top of the P2020 package. The heat sink is designed to withstand shock and vibration tests according to the environmental considerations. The heat sink keeps the processor die temperature below the maximum rating of 125°C under any conditions listed in
Environmental Considerations on page 51 .
ATCA-9405 Installation and Use (6806800M71F) 85
Service Processor
86 ATCA-9405 Installation and Use (6806800M71F)
Chapter 5
Ethernet Infrastructure
5.1
Overview
The Ethernet infrastructure on ATCA-9405 provides switching between any on-board Ethernet port, the ATCA base and fabric interface, and the Ethernet ports provided through RTM.
Separate networks for control and data planes are implemented using VLAN and port based filters, inside the Ethernet switch device.
Table 5-1 Data Plane, Control Plane, and Other Interfaces
Data Plane (DP)
40 GbE connection to redundant Fabric Interface
60 GbE connection to each
Packet Processor (PP)
1 GbE connection to Service
Processor (SP)
160 GbE connection to RTM
20 GbE connection to Update
Channel
Control Plane (CP)
1 GbE connection to redundant base interface
1 GbE connection to each
Packet Processor (PP)
1 GbE connection to Service
Processor (SP)
Others
1 GbE connection from each
Packet Processor (PP) to Face
Plate
1 GbE connection from Service
Processor (SP) to Face Plate
Dual 1 GbE connection to RTM
ATCA-9405 Installation and Use (6806800M71F) 87
Ethernet Infrastructure
Figure 5-1 Ethernet Infrastructure
88
Double data rate XAUI (DXAUI) is 20Gbps interface between Marvell Switch and Cavium CPU, using four SerDes lanes running at 6.25Gbps.
ATCA-9405 Installation and Use (6806800M71F)
Ethernet Infrastructure
5.2
Ethernet Switch
The Ethernet switching device used on ATCA-9405 is Marvell 98CX8234, a highly integrated
32-Port 10 Gigabit Ethernet L2+ Packet Processor with 40 Gigabit Ethernet Uplinks.
The 98CX8234 is a member of the Prestera ® -CX family that addresses bandwidth demand, high density 10G solutions, and support for future 40G upgradability. The device is a packet processor that is managed with an external CPU connected through PCI Express interface. The device supports wide and flexible configuration of network interfaces from 1 GbE to 40 GbE ports.
The ports can operate in Serial Gigabit Media Independent Interface (SGMII), 10 Gigabit
Attachment Unit Interface(XAUI), Reduced 10 Gigabit Attachment Unit Interface (RXAUI), or
40 Gigabit Attachment Unit Interface (XLAUI) Ethernet interface mode. The device has large on-chip memory to support full wire speed L2, L3, and L4 filter performance.
ATCA-9405 Installation and Use (6806800M71F) 89
Ethernet Infrastructure
5.2.1
Port Configuration
Table 5-2 lists all 98CX8234 ports by number, the device to which the port is connected, and
the connection type utilized.
Table 5-2 Switch Port Assignment
Port
Group
0
5
6
7
8-11
3
4
1
2
SerDes
Pair
0
Port
XP0
XP2
XRP4
XRP5
XRP6
XRP7
XLG10
Destination
Packet Processor 2, QLM2,
Data Plane
Packet Processor 2, QLM4,
Data Plane
Service Processor
Packet Processor 2
Packet Processor 2, QLM0,
Data Plane
Fabric Channel 2 - Data Plane
Interface Type
20G, DXAUI
20G, DXAUI
PHY
1G, SGMII
1G, SGMII-RGMII
10G, RXAUI
10G, RXAUI
1000Base-BX (1G)
10GBase-KX4 (10G)
10GBase-KR (10G)
10GBase-KR4 (40G)
88E1512
88X2241
90 ATCA-9405 Installation and Use (6806800M71F)
Ethernet Infrastructure
Table 5-2 Switch Port Assignment (continued)
Port
Group
1
17
18
19
20
21
SerDes
Pair
16
Port
XP16
XP18
XRP20
XRP21
Destination
Packet Processor 1, QLM4,
Data Plane
Packet Processor 1, QLM2,
Data Plane
2
22
23
24-27
35
36
37
38
32
33
34
39
40-43
XRP22
XRP23
XLG26
XPR32
XRP33
XRP34
XRP35
XRP36
XRP37
XP38
XLG42
Interface Type
20G, DXAUI
20G, DXAUI
Packet Processor 1
Packet Processor 1+2 Front
Ethernet
Packet Processor 1, QLM0,
Data Plane
1G, SGMII-RGMII
1G, SGMII
10G, RXAUI
10G, RXAUI
Fabric Channel 1 - Data Plane 1000Base-BX (1G)
10GBase-KX4 (10G)
10GBase-KR (10G)
10GBase-KR4 (40G)
1G, SGMII Serial Redirection
Service Processor 1G, SGMII
Base Channel 1 - Control Plane 1G, SGMII
Base Channel 2 - Control Plane 1G, SGMII
RTM Uplink 1G, SGMII
RTM Uplink
Update Channel - Data Plane
1G, SGMII
20G, DXAUI
RTM Uplink, Data Plane, PHY#3 4x RXAUI (10G)
1x XLG (40G)
PHY
88E1512
88E1512
88X2241
88E1512
88E1322
88E1322
SFP
SFP
NLP10142
ATCA-9405 Installation and Use (6806800M71F) 91
Ethernet Infrastructure
Table 5-2 Switch Port Assignment (continued)
Port
Group
3
49
50
51
52
SerDes
Pair
48
53
54
55
56-59
Port
XRP48
XRP49
XRP50
XRP51
XRP52
XRP53
XRP54
XRP55
XLG58
Destination Interface Type
RTM Uplink, Data Plane, PHY#0 10G, RXAUI
10G, RXAUI
10G, RXAUI
10G, RXAUI
RTM Uplink, Data Plane, PHY#1 10G, RXAUI
10G, RXAUI
10G, RXAUI
10G, RXAUI
RTM Uplink, Data Plane, PHY#2 4x RXAUI (10G)
1x XLG (40G)
PHY
NLP10142
NLP10142
NLP10142
5.2.2
Two-Wire Serial Interface
The 98CX8234 includes a two-wire serial interface (TWSI) for each of the four port groups of the switch. The TWSI supports master/slave transactions and multi master environments
(clock synchronization, interface arbitration). The primary use of the TWSI interface is for switch initialization after reset, when the TWSI operates as a master for serial ROM initialization of the switch. On ATCA-9405, the TWSI interface is not used and switch is fully configured via
Switch Management Interface.
5.2.3
Switch Management Interface
The 98CX8234 provides four PCI Express interfaces for setup, configuration, maintenance, and management by the external SP. Each PCI Express interface is associated to one of the four port groups of the switch. The PCI Express interfaces are connected to PEX8608 PCI Express Switch, as listed in
.
92 ATCA-9405 Installation and Use (6806800M71F)
Ethernet Infrastructure
5.2.4
PHY Management Interface
The 98CX8234 includes three separate serial management interfaces (SMI) for management access to the switch and attached PHY devices. The SMI interface usage and PHY device
connection are listed in Table 5-3 .
The Switch contains four, IEEE 802.3 clause 22 compliant, Master SMI Interfaces (MSMI) for managing external Gigabit Ethernet PHY devices. Each MSMI is associated to one of the four port groups of the switch. This allows the SP to configure the PHY devices and read back PHY status information via read/write access through 98CX8234.
Table 5-3 Gigabit Ethernet PHY Management Interface
Used Interface
MSMI 0
MSMI 1
MSMI 2
Connected PHY
Not used
88E1512
88E1512
SFP
SFP
88E1322
Comment PHY Address
Packet Processor 1+2 Front Ethernet 0x00
Serial Redirection 0x00
SFP Module 1 on ARTM
SFP Module 2 on ARTM
0x01
0x02
Base Channel 1
Base Channel 2
0x11
0x13
MSMI 3 Not used
The Switch contains four port groups, Master SMI Interfaces (MSMI) and IEEE 802.3 clause 45 compliant for managing external 10 Gigabit Ethernet PHY devices that are connected to the
XAUI ports. Each XSMI is associated to one of the four port groups of the switch.
Table 5-4 10G Ethernet PHY Management Interface
Used Interface Connected PHY Comment
MXSMI 0 88X2241 Fabric Channel 2 - Data Plane
MXSMI 1
MXSMI 2
88X2241
NLP10142
Fabric Channel 1 - Data Plane
PHY Address
0x00
0x00
RTM Uplink, Data Plane, PHY#3 0x00 - 0x03
ATCA-9405 Installation and Use (6806800M71F) 93
Ethernet Infrastructure
Table 5-4 10G Ethernet PHY Management Interface (continued)
Used Interface Connected PHY Comment
MXSMI 3 NLP10142
PHY Address
RTM Uplink, Data Plane, PHY#0 0x00 - 0x03
NLP10142
NLP10142
RTM Uplink, Data Plane, PHY#1 0x04 – 0x07
RTM Uplink, Data Plane, PHY#2 0x08 – 0x0B
The four Slave SMI Interfaces (CPU_SMI) for host access to all address mapped entities in the
Switch are not used on ATCA-9405. The PCI Express interface is used instead.
5.3
Base Interface
The redundant (dual) ATCA base interface is provided by the Ethernet Switch. The physical
1000Base-T interface (according to PICMG3.0) is implemented using one Marvell 88E1322
Integrated 10/100/1000 Dual Gigabit Ethernet Transceiver.
Two SGMII lanes of the Ethernet Switch are connected to the PHY device. Basic Ethernet PHY configuration after power-up is done using hardware strapping options. After power-up the
PHY is managed by the Switch via MII interface.
5.4
Fabric Interface
The redundant (dual) ATCA fabric interface is provided by the Ethernet Switch. The physical interface is implemented using two Marvell 88X2241 10Gbps Quad Channel Transceivers.
The two 88X2241 Transceivers provide eight ports (4 ports each) that are directly connected to Zone 2 connector P23 row 4+3 (Fabric Channel 1, Port 0-3) and connector P23 row 2+1
(Fabric Channel 2, Port 0-3).
The 88X2241 devices are connected to the Ethernet Switch using ports XLG10 and XLG26. For more information, refer
Configuration of the 88X2241 device is done via its MDIO/MDC interface which is connected to the Ethernet Switch. Fabric Channel 1 Transceiver is connected to MXSMI1 (XLG26) interface. Fabric Channel 2 Transceiver is connected to MXSMI0 (XLG10) interface.
94 ATCA-9405 Installation and Use (6806800M71F)
Ethernet Infrastructure
The following Ethernet interconnection options are supported by ATCA-9405.
Figure 5-2 Fabric Interface Mode 1x 1000Base-KX (PICMG3.1 Option 1)
Figure 5-3 Fabric Interface Mode 1x 10GBase-KX4 (PICMG3.1 Option 9)
ATCA-9405 Installation and Use (6806800M71F) 95
Ethernet Infrastructure
Figure 5-4 Fabric Interface Mode 4x 10GBase-KR
Figure 5-5 Fabric Interface Mode 1x 40GBase-KR4
96 ATCA-9405 Installation and Use (6806800M71F)
Ethernet Infrastructure
5.5
Update Channel
The Update Channel Interface is provided by the Ethernet Switch and is physically implemented using one DXAUI port. The DXAUI port is directly connected to Zone 2 connector P20 row 4
(Update Channel Port 0-1) and connector P20 row 3 (Update Channel Port 2-3). E-Keying for the Update Channel is guaranteed via 98CX8234 Ethernet Switch, which disables its ports after power-up reset and enables individual ports only by configuration.
Figure 5-6 Update Channel
5.6
Serial Redirection
Terminal Server is implemented to provide serial redirection into base network for serial consoles of SP and both Packet Processors. The Terminal Server functionality is implemented using routing logic implemented in the Glue Logic FPGA plus an additional Microcontroller.
The Microcontroller converts the incoming UART ports from Glue Logic FPGA into an Ethernet interface, which is connected to the Ethernet Switch. For the connection between
Microcontroller and Switch, Marvell 88E1512, Gigabit Ethernet Transceiver in 100Base-T mode is used.
ATCA-9405 Installation and Use (6806800M71F) 97
Ethernet Infrastructure
The Ethernet Switch is configured at power up via serial EEPROM (refer
Figure 5-7 Serial Redirection
98 ATCA-9405 Installation and Use (6806800M71F)
Chapter 6
Service Infrastructure
6.1
Overview
The Service Processor (SP) monitors and controls the infrastructure of blade and RTM through high speed PCIe interfaces. Connection between the PCIe root complex and PCIe endpoints is provided through PEX8608, a fully non-blocking, low latency, and low power 8-lane, 8-port
PCI-Express Gen 2 switch.
The PEX8608 provides up to eight configurable PCIe ports with x1or x2 link width (number of lanes per unique link). The port numbers are 0, 1, and 4-9 as defined in
ports can be designated (or dynamically changed) to be the upstream port, that provides connection towards the PCIe root complex.
Link width auto negotiation, dynamic lane reversal, and polarity reversal during link training process is supported for all ports. Other features of the switch are:
Integrated 2.5 or 5.0 GT/s SerDes speed negotiation per port
Non-blocking crossbar switch architecture
Low packet latency and high performance
PCIe power management
Quality of service, reliability, availability, and serviceability features
Out-of-Band initialization options
JTAG support
6.2
Port Configuration
Port configuration is a part of the PEX8608 initialization process that starts upon exiting from the fundamental reset through PEX_PERST# reset input. The configuration of upstream port, port width, and other hardware modes are initially set by hardware strapping signals.
ATCA-9405 Installation and Use (6806800M71F) 99
Service Infrastructure
The physical layer device (PHY) of the configured ports attempt to bring up the links, which includes link training process, link initialization, and automatic link width negotiation.
The ports that are neither configured nor enabled are invisible to the software.
The following table lists the port configuration:
Table 6-1 PEX8608 Port Configuration
Port
Number
Port 0
Port 1
Port 4
Port 5
Port 6
Port 7
Port 8
Port 9
6
3
7
5
2
4
1
Lane
Number
0 x1 x1 x1 x1 x1 x1 x1
Link
Width x1
Root
Complex x
Destination
Service Processor P2020
Packet Processor 1 - CN6880
Packet Processor 2 - CN6880
Ethernet Switch 98CX8234, Port Group 1
Ethernet Switch 98CX8234, Port Group 2
Ethernet Switch 98CX8234, Port Group 3
Ethernet Switch 98CX8234, Port Group 4
RTM Interface
The SP has the role of PCIe root complex, thus Port 0 is configured as default upstream port.
6.3
Hot Plug Support
ATCA-9405 supports software controlled power-down and power-on sequence for the two PP units including memory, Gigabit Ethernet PHY, TCAM module, and all external interfaces. This is used to reduce power consumption of the blade, when application does not need full computing performance of both PPs.
100 ATCA-9405 Installation and Use (6806800M71F)
Service Infrastructure
The application running on the SP is responsible for the power cycle. The two Packet Processor
(PP) units are connected to the SP through PCIe interface, thus the power-down and power-on sequence is handled like a Hot Plug event for standard PCIe plug-in card, insertion and removal process.
Hot Plug is supported for PCIe Port 1 (PP 1) and PCIe Port 4 (PP 2). The implementation is compliant with PCI Hot-Plug Specification [14] and Standard Hot-Plug Controller and Subsystem
Specification [15].
6.3.1
Serial Hot Plug Controller
The PEX8608 supports insertion and removal for all downstream ports through an I2C based
Serial Hot Plug Controller (SHPC) and an external I/O Expander per port. The SHPC is able to control the ports of the I/O Expander through I2C master interface and retrieve the port status, such as device connect status, power fault, or MRL sensor position.
Figure 6-1 PEX8608 Serial Hot Plug Controller
The application running on the SP is responsible for interaction with the SHPC and the additional logic in the Power CPLD and Glue Logic FPGA. The two I/O Expander are implemented in the Glue Logic FPGA.
ATCA-9405 Installation and Use (6806800M71F) 101
Service Infrastructure
6.4
I2C Slave Interface
PEX8608 includes an I2C slave interface. The interface is a sideband mechanism that allows the device configuration registers to be programmed, independent of the PCIe upstream link.
The I2C slave interface is connected to second I2C interface of P2020. For more information,
see I2C Interface on page 83 .
6.5
JTAG Support
The IEEE 1149.1 compliant JTAG boundary scan interface of PEX8608 is connected to the onboard payload JTAG chain.
6.6
Lane Status
Lane status outputs are provided. Each output is directly connected to a small LED to provide visual indication regarding the PHY of each lane. LEDs are encoded as listed below:
LED offLane is disabled
LED onLane is enabled, 5.0 GT/s
LED blinkingLane is enabled, 2.5 GT/s
Table 6-2 Lane Mapping
D22
D23
D24
D25
D26
D27
D28
D29
RTM
Ethernet Switch Port Group 3
Ethernet Switch Port Group 1
Packet Processor 1
Ethernet Switch Port Group 4
Ethernet Switch Port Group 2
Packet Processor 1
Service Processor
102 ATCA-9405 Installation and Use (6806800M71F)
Figure 6-2 PEX8608 Lane Status LEDs
Service Infrastructure
ATCA-9405 Installation and Use (6806800M71F) 103
Service Infrastructure
104 ATCA-9405 Installation and Use (6806800M71F)
Chapter 7
Mezzanine Module
7.1
Overview
ATCA-9405 provides a flexible mezzanine module based design approach to design and plug in various modules that support or add features for the Packet Processors units, for example:
Module with TCAM processor to offload the Packet Processors
Module for Interlaken interconnection between Packet Processor 1 and 2. This module is the default assembly option for standard ATCA-9405 variants.
Module for x4 PCI Express Gen 2 connection toward Zone 3 to enable design of custom
ARTM for mass storage.
Other modules possible on customer request.
ATCA-9405 Installation and Use (6806800M71F) 105
Mezzanine Module
The following block diagram shows ATCA-9405 with TCAM Module plugged:
Figure 7-1 Block Diagram with TCAM Module
106 ATCA-9405 Installation and Use (6806800M71F)
Mezzanine Module
The following figure shows ATCA-9405 with PCI-Express module plugged:
Figure 7-2 Block Diagram with PCI-Express Module
ATCA-9405 Installation and Use (6806800M71F) 107
Mezzanine Module
TCAM and PCI-Express module are optional items. Contact sales/marketing team, if you require a mezzanine module.
108 ATCA-9405 Installation and Use (6806800M71F)
Chapter 8
Intelligent Peripheral Management Controller
8.1
Overview
The ATCA-9405 provides an intelligent hardware management system, as defined in the
AdvancedTCA® Base Specification (PICMG® 3.0; AMC.0). This system incorporates two IPMI controllers:
An Intelligent Platform Management Controller (IPMC) based on the BMR-H8S-AMCc® reference design from Pigeon Point Systems.
A Module Management Controller (MMC) residing at the RTM based on the BMR-AVR-
AMCm® reference design from Pigeon Point Systems.
Pigeon Point Systems IPM Sentry products are consistent with all current PICMG specifications as well as IPMI v1.5 compliant with specific 2.0 extensions.
8.2
Functional Overview
The ATCA-9405 implements all the standard Intelligent Platform Management Interface (IPMI) commands and provides hardware interfaces for other system management features such as
Hot Swap control, LED control, power control, and temperature and voltage monitoring. The
IPMC also supports a Keyboard Controller Style (KCS) based host interface for direct payload to
IPMI communication.
The ATCA-9405 provides the following features:
HPM.1 firmware upgrades and crisis recovery
IPMI messaging
IPMI channels and sessions
Sensor Data Records and SDR Repository
FRU Inventory
Sensors
POST
Asynchronous Event Notification
U-boot Boot Configuration Parameter
Payload Boot Bank Selection
ATCA-9405 Installation and Use (6806800M71F) 109
Intelligent Peripheral Management Controller
Serial Line Selection
Built-in Terminal Server
Settable Graceful Shutdown Timeout
IPMI Hardware Watchdog Timer
Fail-safe
Local System Event Log (SEL)
The IPMC at the front board actslike a carrier IPMC. It retrieves the sensor information of the
MMC and creates an SDR repository that provides direct access to all sensors within the system. The IPMC is implemented as the managed FRU #0 and the MMC as FRU #1. All commands which are directed to the MMC are bridged by the IPMC.
The P2020 communicates with the IPMC using the Keyboard Controller Style (KCS) interface of the H8S. The FRU inventory, SEL events, and the SDR information is stored in external I2C
EEPROMS. This enables post-mortem analysis, when the system processor is disabled.
IPMB buffers on both the IPMB-0 busses are used to isolate a faulty IPMB bus from the backplane.
IPMC can access the registers within the FPGA and the Power CPLD via SPI bus. This enhances the capabilities of the IPMC. The FPGA is used to monitor the CPU status, the Payload reset cause, and to control the boot bank selection. The Power CPLD controls the enabling and monitoring of power good signals from all on-board power converters.
The functional block diagram of the ATCA-9405 IPMC/MMC system is shown in Figure 8-1 on page 111 .
110 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Figure 8-1 ATCA-9405 IPMC Block Diagram
ATCA-9405 Installation and Use (6806800M71F) 111
Intelligent Peripheral Management Controller
8.3
Firmware Architecture
The IPMC and MMC firmware basically consists of three major parts:
Boot loader
Hardware Abstraction Layer (HAL)
Application Layer
The boot loader maintains redundant copies of the firmware in flash. Each time the IPMI firmware is upgraded, a redundant copy of the current IPMI firmware is made in flash.
The Hardware Abstraction Layer (HAL) initializes H8S/ATMEL and makes all preparations necessary for running code written in C. The time management facility of the HAL provides a means for measuring time and detecting timeout conditions. The device drivers are responsible for implementing high-level interfaces to the hardware.
The Application layer is implemented as a multi threaded application. The main thread reads incoming messages/events from various inbound queues, processes these messages/events, and produces outgoing traffic to appropriate hardware interfaces.
The IPMC implements a number of subsidiary threads in addition, to serve RTM module discovery and e-keying management.
112 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
The Application layer can also operate in standalone mode intended to debug the payload without requiring a shelf manager.
Figure 8-2 Firmware Architecture
8.4
HPM.1 Components
All embedded software images are upgraded via HPM.1 protocol. The following are the HPM.1 components:
IPMI bootloader
IPMI firmware
IPMI FRU information
U-boot firmware
FPGA
ATCA-9405 Installation and Use (6806800M71F) 113
Intelligent Peripheral Management Controller
The HPM.1 components have different component properties due to its different physical implementation:
Table 8-1 HPM.1 Components
Components ID
IPMI bootloader
Payload cold reset required
1 no
0 yes IPMI firmware
IPMI FRU information
2 yes u-boot firmware
FPGA
5 yes
4 yes (even power cycle) yes no no no
Deferred activation support no
Comparism support no no no no no
Preparation support yes yes yes yes yes
Rollback/backup support no supported without backup command supported without backup command supported without backup command supported without backup command
Component name
IPMI B/L
IPMI F/W
IPMI F/I
PYLD F/W
PYLD FPGA
114 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.4.1
FPGA Firmware Upgrade
The FPGA device is a special HPM.1 component. Physically it consists of one working and one golden image. The working image can be overwritten and the golden image is write-protected.
Thus only the working component can be programmed.
Figure 8-3 FPGA Memory Map
If the FPGA determines that the working image is broken, it automatically selects the golden image to boot. This mechanism is close to HPM.1 automatic rollback. However, manual/automatic rollback can be performed only once (selection of the golden image is possible by writing "foo" data into the working image only. The FPGA bootloader determines the corrupt working image and jumps to the golden image). Note that this is a limitation, manual rollback to the old working image is not possible. Rollback means switching to the golden version.
A payload cold reset is not enough to execute a new FPGA component. In this case, a power cycle is required.
ATCA-9405 Installation and Use (6806800M71F) 115
Intelligent Peripheral Management Controller
8.4.2
Payload Firmware Upgrade
The HPM.1 component u-boot is implemented with two SPI flashes, one active and one backup. The IPMC always writes to the backup flash. Automatic rollback is implemented via fail-safe architecture. For details about failsafe, see
Fail Safe Logic and Watchdog Support on page 142 .
The IPMC always upgrades the backup boot flash as intended. However, the HPM1 command
Activate Firmware
does not reboot the payload firmware unconditionally. Instead the blade can be rebooted gracefully to activate the firmware.
Executing two u-boot firmware upgrades without a payload reset does not upgrade both flashes for security reason (just the backup flash can be programmed).
Crisis recovery is fully supported. Two broken u-boot images can be reprogrammed via IPMI with the help of the ShMM.
8.4.3
IPMC Firmware Upgrade
The HPM1 component IPMI firmware stores its active and backup image within one physical flash. A small bootloader is used to either jump to the active or to the backup image depending on the boot flags indicating successful boot. The bootloader is implemented as HPM1 component; however there is no backup image.
The boot loader maintains redundant copies of the firmware in flash. Each time the IPMI firmware is upgraded, a redundant copy of the current IPMI firmware is made in flash. Once the new IPMI firmware is programmed, the IPMI controller resets itself to boot from the new image. The boot loader also validates new IPMI firmware images. If the power up of the IPMI controller is successful, then the actual image is made active and the previously active image is made backup. If the power up fails, the boot loader automatically recovers from crisis by switching to the first firmware image that has booted before.
The IPMI controller can be upgraded via KCS or IPMB interface. To ensure that the payload is not interrupted during IPMI firmware upgrade, the IPMI controller stores all operational information, such as e-keying, hot-swap state, last events to be queued, graceful shutdown timeout, latest pin settings, and so on in non-volatile storage.
116 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.4.4
Manual Rollback
To be able to switch from an active component to its rollback version, the HPM1 command
Initiate Manual Rollback
is enhanced with a 2 request data argument, to enable users to select the component which should be rolledback. This is done to overcome a limitation in the HPM1 specification. Using this command extension, you can initiate the component rollback without previous firmware upgrades.
8.4.5
Retrieving Versioning Information
Retrieving the actual and backup version of the components is possible without switching payload firmware boot banks. We recommend to use fcu for embedded firmware upgrade.
For example, the following terminal output illustrates the component versions: hayabusa(avh012):119 ./fcu -q -t8a --lan=192.168.42.50
********************[[[[[REPORT BEGIN]]]]]********************
OPERATION : Query
RESULT : SUCCESS
MESSAGE : Device : 0065CD-2003-hpm.1-ipmc
Part number : 123456789123456789123
Part revision :
BANK : A - Operational
Firmware Name : IPMI F/W
Firmware Version : 2.0.00000002
BANK : B - Rollback
Firmware Name : IPMI F/W
Firmware Version : 2.0.00000002
BANK : D - Operational
Firmware Name : IPMI B/L
Firmware Version : 2.0.00000002
BANK : G - Operational
Firmware Name : IPMI F/I
Firmware Version : 0.0.00000000
ATCA-9405 Installation and Use (6806800M71F) 117
Intelligent Peripheral Management Controller
BANK : J - Operational
Firmware Name : PYLD FPGA
Firmware Version : 0.40.0000000F
BANK : K - Rollback
Firmware Name : PYLD FPGA
Firmware Version : 0.120.00000010
BANK : M - Operational
Firmware Name : PYLD F/W
Firmware Version : 1.0.00000001
BANK : N - Rollback
Firmware Name : PYLD F/W
Firmware Version : 1.0.0000000A
********************[[[[[ REPORT END ]]]]]********************
8.5
Sensors
This section describes the analog and discrete Sensors available at the ATCA-9405.
Table 8-2 on page 118 , lists the sensor identification numbers and information regarding the
sensor type, name, supported thresholds, assertion and deassertion information, and a brief description of the sensor purpose.
Table 8-2 ATCA-9405 Specific Sensors
Nr
0
Sensor
Name
Hot Swap
Carrier
Sensor Type
Hot Swap
0xF0
Event/
Reading
Type
Sensorspecific discrete
0x6F
Event
Data
Byte 1
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
Event Data
Byte 2
[7:4] = Cause
[3:0] = Previous
State
Event Data
Byte 3
FRU ID
Event
Threshold/
Description
0x0: M0
0x1: M1
0x2: M2
0x3: M3
0x4: M4
0x5: M5
0x6: M6
0x7: M7
Assertion
Deassertion Rearm
Assertion Auto
118 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
1
Sensor
Name Sensor Type
HS_ARTM Hot Swap
0xF0
2
3
Version change
IPMB
Physical
Version
Change
0x2B
Physical
IPMB-0
0xF1
Event/
Reading
Type
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
0x0
0x1
0x2
0x3
Event
Data
Byte 1
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
0x7
Event Data
Byte 2
[7:4] = Cause
[3:0] = Previous
State
Change type
[7:4] = Channel
Number
[3:0] = Reserved
Event Data
Byte 3
FRU ID
0xFF reading
5 12.0V
6 +5.0V
7 3.3V
8
9
3.3V
Mgmt
1.2V
10 1.05V
11 1.0V
Switch
4 BMC
Watchdog
Watchdog 2
0x23
Sensorspecific discrete
0x6F
Voltage
0x02
Voltage
0x02
Voltage
0x02
Voltage
0x02
Voltage
0x02
Voltage
0x02
Voltage
0x02
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
0x0
0x1
0x2
0x3
0x8
See IPMI Spec reading reading reading reading reading reading reading
0xFF threshold threshold threshold threshold threshold threshold threshold
Event
Threshold/
Description
0x0: M0
0x1: M1
0x2: M2
0x3: M3
0x4: M4
0x5: M5
0x6: M6
0x7: M7
0x7: Software or F/W change successful
Assertion
Deassertion Rearm
Assertion Auto
Assertion Auto
0x0: IPMB-A disabled,
IPMB-B disabled
0x1: IPMB-A enabled,
IPMB-B disabled
0x2: IPMB-A disabled,
IPMB-B enabled
0x3: IPMB-A enabled,
IPMB-B enabled
0x0: Timer expired
0x1: Hard Reset
0x2: Power Down
0x3: Power Cycle
0x8: Timer Interrupt unr uc lnr lc
Assertion
Assertion unr uc lnr lc unr uc lnr lc unr uc lnr lc unr uc lnr lc unr uc lnr lc unr uc lnr lc
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Auto
Auto
Auto
Auto
Auto
Auto
Auto
Auto
Auto
ATCA-9405 Installation and Use (6806800M71F) 119
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
12 1.0V
13
14 switch temp
P2020 temp
Sensor Type
Voltage
0x02
Temp
0x01
Temp
0x01
15 outlet temp
Temp
0x01
16 inlet temp Temp
0x01
17 IPMC
POST
18 BootBank
Management
Subsystem
Health
0x28
OEM
0xD2
19 Fw Prog
SP
System
Firmware
Progress
0x0F
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
Event/
Reading
Type
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01 digital
Discrete
0x06
Event
Data
Byte 1
Event Data
Byte 2 reading
0x0
0x1
0x0
0x0
0x1
0x2 reading reading reading reading
0xFF
0xFF
See IPMI Spec
20 OS Boot
SP
OS Boot
0x1F
Sensorspecific discrete
0x6F
0x0
0x1
0x2
0x3
0x4
0x5
0x6
0xFF
Event Data
Byte 3 threshold threshold threshold threshold threshold
0xFF
0xFF
Event
Threshold/
Description unr uc lnr lc unr uc lnr lc unr uc lnr lc unr uc lnr lc unr uc lnr lc
0x0: Performance Met
0x1: Performance
Lags
Assertion
Deassertion Rearm
Assertion/
Deassertion
Auto
Auto Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion
Auto
Auto
Auto
Auto
0x0: Boot Bank A Assertion Auto
See IPMI Spec
0xFF
0x0: System Firmware
Error
0x1: System Firmware
Hang
0x2: System Firmware
Progress
0x0: A: boot completed
0x1: C: boot completed
0x2: PXE boot completed
0x3: Diagnostic boot completed
0x4: CD_ROM boot completed
0x5: ROM boot completed
0x6: boot completed
Assertion
Assertion
Auto
Auto
120 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
21 Boot Err
SP
Sensor Type
Boot Error
0x1E
Event/
Reading
Type
Sensorspecific discrete
0x6F
Event
Data
Byte 1
0x0
0x1
0x2
0x3
0x4
Event Data
Byte 2
0xFF
22 ATCA-
9405
IPMC
OEM
0xD5
Sensorspecific discrete
0x6F
0x0
0x1
0x2
0x3
0x4
0x5
0x0
0x1
See IPMI Spec 23 Power
Good
24 Reset
Source
Power Supply
0x08
OEM
0xDA
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
0x0 0x00
25 PP #0
DDR1 temp
26 PP #0
DDR2 temp
27 PP #0
DDR3 temp
Temp
0x01
Temp
0x01
Temp
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
ATCA-9405 Installation and Use (6806800M71F) reading reading reading
Event Data
Byte 3
0xFF
0xFF
[7] = Paload
IPMC reset
[6] = reserved
[5] = SPP
Hreset
[4] = Push
Button Reset
RTM
[3] = SW Prog
Watchdog
Reset
[2] = Pus
Button Reset
[1] = reserved
[0] = Power
GOOD Reset threshold unr uc unc
Event
Threshold/
Description
0x0: No Bootable media
0x1: Non-bootable diskette
0x2: PXE Server not found
0x3: Invalid boot sector
0x4: Timout waiting for user selection
0x0: Watchdog Reset
0x1: Software Reset
0x2: Power Failure
0x3: Hard Boot
0x4: Cold Boot
0x5: Warm Boot
0x0: Presence detected
0x1: Power Supply
Failure detected
0x0: Payload Reset detected. Cause delivered in Event
Byte 2/3
Assertion
Deassertion Rearm
Assertion Auto
Assertion
Assertion
Assertion
Auto
Auto
Auto
Assertion/
Deassertion
Auto threshold unr uc unc threshold unr uc unc
Assertion/
Deassertion
Auto
Assertion/
Deassertion
Auto
121
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
28 PP #0
DDR4 temp
29 PP #1
DDR1 temp
30 PP #1
DDR2 temp
31 PP #1
DDR3 temp
32 PP #1
DDR4 temp
33 SP CPU
Status
Sensor Type
Temp
0x01
Temp
0x01
Temp
0x01
Temp
0x01
Temp
0x01
Processor
0x07
Event/
Reading
Type
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Sensorspecific discrete
0x6F
Event
Data
Byte 1
Event Data
Byte 2 reading
0x7
0x8
0xA reading reading reading reading
0xFF
34 PP #0 CPU
Status
Processor
0x07
Sensorspecific discrete
0x6F
0x1
0x7
0x8
0xA
0xFF
35 PP #1 CPU
Status
Processor
0x07
36
37
38
FPGA
Status
-48v A
Volts
-48v B
Volts
Processor
0x07
Voltage
0x02
Voltage
0x02
Sensorspecific discrete
0x6F
0x1
0x7
0x8
0xA
0xFF
Sensorspecific discrete
0x6F
Threshold
0x01
Threshold
0x01
0x7
0x8
0xB
0xFF reading reading
Event Data
Byte 3 threshold threshold threshold threshold threshold
0xFF
0xFF
0xFF
0xFF threshold threshold
Event
Threshold/
Description unr uc unc unr uc unc unr uc unc unr uc unc unr uc unc
0x7: Processor
Presence detected
0x8: Processor disabled
0xA: ProcHot
0x1: Thermal Trip
0x7: Processor
Presence detected
0x8: Processor disabled
0xA: ProcHot
0x1: Thermal Trip
0x7: Processor
Presence detected
0x8: Processor disabled
0xA: ProcHot
0x7: Processor
Presence detected
0x8: Processor disabled
0xB: Conf CRC Error unr uc lnr lc unr uc lnr lc
Assertion
Deassertion Rearm
Assertion/
Deassertion
Auto
Assertion/
Deassertion
Auto
Assertion/
Deassertion
Auto
Assertion/
Deassertion
Auto
Assertion/
Deassertion
Auto
Assertion Auto
Assertion
Assertion
Assertion
Auto
Auto
Auto
Assertion/
Deassertion
Assertion/
Deassertion
Auto
Auto
122 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
39 -48v
Amps
40 HoldUp
Cap Volts
41 PWR
Entry
Temp
42 PWR
Entry
Status
Sensor Type
Current
0x03
Voltage
0x02
Temp
0x01
OEM
0xD7
Event/
Reading
Type
Threshold
0x01
Threshold
0x01
Threshold
0x01
Sensorspecific discrete
0x6F
Event
Data
Byte 1
Event Data
Byte 2 reading
0x0 reading reading
43 FPGA
Watchdog
OEM
0xCF
Sensorspecific discrete
0x6F
0x1
Event Data
Byte 3 threshold threshold threshold
Event
Threshold/
Description
No Thresholds unr uc lnr lc unr uc unc
Synchor Pwr Entr
Module:
[6] = VOUT_low
[5] = Hotswap
[4] = Holdup
[2] = Alarm
[1] = Enable_B
[0] Enable_A
Emerson Pwr Entry
Module:
[7] = DIG_Fault
[6] =
HUCapEngage
[5] =
Hotswap_Enable
[4] =
HUCap_Switch
[3] =
Alarm_Control
[1] = DIG_Alarm
[0] =
Sec_MCU_Fault
All other bits are reserved
See IPMI Spec 0xFF
[7:6] = Pwr
Entry Module
0 = Synchor
1 = Emerson
Emerson Pwr
Entry Module:
[2] =
DIG_EnableA
[1] =
DIG_EnableB
[0] =
Mcu_Fault
All other bits are reserved
0x0: Pwr Entry
Module Status
Change detected
0x1: Hard Reset
Assertion
Deassertion Rearm
Auto
Assertion/
Deassertion
Assertion/
Deassertion
Auto
Auto
Assertion Auto
Assertion Auto
ATCA-9405 Installation and Use (6806800M71F) 123
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
44 CPLD
PwrF SP
Sensor Type
OEM
0xE0
Event/
Reading
Type
Sensorspecific discrete
0x6F
Event
Data
Byte 1
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
Event Data
Byte 2
45 CPLD
PwrF PP0
OEM
0xE1
46 CPLD
PwrF PP1
OEM
0xE2
Sensorspecific discrete
0x6F
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
Sensorspecific discrete
0x6F
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
Event Data
Byte 3
Event
Threshold/
Description
0x0: 12V Power Good
0x1: 5.0V/3.3V Power
Good
0x2: 2.5V/1.0V Power
Good
0x3: 1.8V/0.9V Power
Good
0x4: 1.0 Power Good
Switch
0x5: 1.2 Power Good
FPGA
0x6: 1.5V/1.05V
Power Good ETH SP
0x7: reserved
0x0: 3.3V Power Good
PP2
0x1: 2.5V Power Good
PP2
0x2: 1.00V Power
Good GPP PP2
0x3: VQLM Power
Good PP2
0x4: VDDR Power
Good PP2
0x5: reserved
0x6: Reserved
0x7: reserved
0x0: 3.3V Power Good
PP1
0x1: 2.5V Power Good
PP1
0x2: 1.00V Power
Good GPP PP1
0x3: VQLM Power
Good PP1
0x4: VDDR Power
Good PP1
0x5: reserved
0x6: Reserved
0x7: reserved
Assertion
Deassertion Rearm
Assertion Auto
Assertion
Assertion
Auto
Auto
124 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
47 CPLD
PwrF
IPMC
48 CPLD
PwrF
FPGA
Sensor Type
OEM
0xE3
OEM
0xE4
Event/
Reading
Type
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
Event
Data
Byte 1
0x4
0x5
0x6
0x7
0x0
0x1
0x2
0x3
Event Data
Byte 2
0x0
0x1
0x2
0x3
0x4
0x5
0x6
0x7
49 ADT7461
#0 temp
50 PP #0
Temp
51 ADT7461
#1 temp
52 PP #1
Temp
53 48V A
Supply
54 48V B
Supply
Temp
0x01
Temp
0x01
Temp
0x01
Temp
0x01
Power Supply
0x08
Power Supply
0x08
Threshold
0x01
Threshold
0x01
Threshold
0x01
Threshold
0x01
Sensorspecific discrete
0x6F
Sensorspecific discrete
0x6F
0x0
0x1
0x0
0x1 reading reading reading reading
See IPMI Spec
See IPMI Spec threshold threshold threshold threshold
0xFF
Event Data
Byte 3
0xFF
Event
Threshold/
Description
0x0: RTM Power
Good
0x1: RTM MGMT
Power Good
0x2: RTM PP Power
Good
0x3: RTM PP Fault
0x4: TCAM Power
Good
0x5: reserved
0x6: Reserved
0x7: reserved
0x0: Glue Logic FPGA
Done
0x1: Glue Logic FPGA
Init
0x2: CONC CRC ERR
0x3: FPGA Done RTM
0x4: FPGA INIT_N
RTM
0x5: CONF CRC ERR
RTM
0x6: Reserved
0x7: reserved unr uc unc
Assertion
Deassertion Rearm
Assertion Auto
Assertion Auto
Auto unr uc unc unr uc unc unr uc unc
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Assertion/
Deassertion
Auto
Auto
Auto
Auto 0x0: Presence detected
0x1: Power Supply
Failure detected
0x0: Presence detected
0x1: Power Supply
Failure detected
Assertion/
Deassertion
Auto
ATCA-9405 Installation and Use (6806800M71F) 125
Intelligent Peripheral Management Controller
Table 8-2 ATCA-9405 Specific Sensors (continued)
Nr
Sensor
Name
55 Fw Prog
PP #0
Sensor Type
System
Firmware
Progress
0x0F
Event/
Reading
Type
Sensorspecific discrete
0x6F
Event
Data
Byte 1
0x0
0x1
0x2
Event Data
Byte 2
See IPMI Spec
56 Fw Prog
PP #1
System
Firmware
Progress
0x0F
Sensorspecific discrete
0x6F
0x0
0x1
0x2
See IPMI Spec
Event Data
Byte 3
See IPMI Spec
See IPMI Spec
Event
Threshold/
Description
0x0: System Firmware
Error
0x1: System Firmware
Hang
0x2: System Firmware
Progress
0x0: System Firmware
Error
0x1: System Firmware
Hang
0x2: System Firmware
Progress
Assertion
Deassertion Rearm
Assertion Auto
Assertion Auto
8.5.1
Firmware Progress, OS Boot, and Boot Error Sensor
The IPMC firmware provides a Firmware Progress, OS Boot, and Boot Error Sensor to enable SP payload firmware and SP payload OS to report boot progress and OS Boot via IPMI event messages.
The firmware progress sensor is of type 0x0F (System Firmware Progress) and is used to pass payload status information to the IPMC, which is then logged to the SEL (both local and remote). While the payload is booting, the payload logs events to the Firmware Progress
Sensor to indicate the progress of the boot process.
The boot error sensor is of type 0x1E (Boot Error) and is used to pass boot failure information to the IPMC, which is then logged to the SEL.
The OS Boot sensor is of type 0x1F (OS Boot) and is used to inform the IPMC when the operating system has completed its boot up sequence.
8.5.2
Boot Bank Supervision Sensor
The boot bank supervision sensor is intended to always give the actual boot bank, from which the payload has booted last. The boot bank information received from this sensor may differ from the boot bank selection performed, in case if the boot bank selection has changed after the payload has booted.
126 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.5.3
POST Results Sensor
The POST results sensor is of type 0x28 (Management subsystem health), which returns information on whether the power-on self-test (POST) of the IPMI firmware passes or not. For more information on POST, see
8.5.4
Power Good Sensor
The Payload Power sensor is of type 0x08 (Power Supply) which reports the state of the payload power. Since the Digital Power Monitor can disable payload power on its own due to a faulty
DC-DC converter, the IPMC monitors the health and reports an event when the Digital Power
Monitor disables power on its own. As a result, the IPMC proactively transitions the IPMC from
M4 to M6 (to M1) with a cause code of 0x09 (Unexpected Deactivation). This automatic shutdown is meant to keep the IPMCs state in-line with the payload state.
8.5.5
Power Interface Sensors
A PowerOne Power input module (Synqor) is used for the 48v to 12v conversion monitoring.
The Synqor includes sensors which monitor the shelf's 48v input feeds current, hold-up capacitor voltage, on-board temperature, and internal status. The status sensor reading can be
decoded as shown in Table 8-3 on page 127 .
Table 8-3 Status Sensor's Sensor Reading
Bit 7
Bit 6
Bit 5
Bit 4
Bit 3
Bit 2 reserved 0
-48v Output Under
Voltage Alarm
0: Output Voltage is below threshold
1: Output Voltage is above threshold
Hot-swap Switch State 0: Hot-swap switch is off
1: Hot-swap switch is on
Holdup Switch State 0: Holdup Cap is not connected to -48V
Out
1: Holdup Cap is connected to -48V Out
Reserved
Alarm Signal State
0
0: Primary side Alarm is not set
1: Primary side Alarm is set
ATCA-9405 Installation and Use (6806800M71F) 127
Intelligent Peripheral Management Controller
Table 8-3 Status Sensor's Sensor Reading (continued)
Bit 1
Bit 0
Voltage Feed B Enabled 0: Enable B is Disabled
1: Enable B is Enabled
Voltage Feed A
Enabled
0: Enable A is Disabled
1: Enable A is Enabled
8.5.6
Reset Cause Sensor
This sensor is implemented to monitor the last payload reset cause, such as hard reset, front panel reset, and so on. The IPMC evaluates the Reset Cause Register within the Glue Logic
FPGA.
8.5.7
Presence Sensors
These sensors are implemented for each SFP as well as for the cpus and the FPGAs. These sensors provide the presence and absence status. Also, the PP #1 and PP #0 specific sensor can assert TEMP ALERT, the FPGAs can assert CONF_ERROR.
8.5.8
Voltage and Temperature Sensors
Voltage and temperature sensors are available at the front blade and at the RTM.
Table 8-4 Voltage and Temperature Sensor Devices
I2C address I2C bus
0x5E IPMC private
Domain
Front blade
0x92
0x96
0x9C
IPMC private
IPMC private
IPMC private
Front blade
Front blade
Front blade
Purpose
48V Feed A
48V Feed B
Current
Holdup
Temp
Status
Temperature Switch
Temperature P2020
Temperature outlet
Device
Power Entry Module
LM75
LM75
LM75
128 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-4 Voltage and Temperature Sensor Devices (continued)
I2C address I2C bus
0x9E IPMC private
Int ADC IPMC private
0x98
0x98
0xA0
0xA2
0xA4
0xA6
0xA0
0xA2
0xA4
0xA6
0x90
0x92
0x94
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
IPMC private2
MMC
MMC
MMC
Domain
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
Front blade
RTM
RTM
RTM
Purpose
Temperature inlet
Device
LM75
12V
5V
3.3V Mgmt
3.3V
1.2V
1.05V
1.0 V Switch
1.0V
Temperature cpu PP0
H8S ADC
Temperature cpu PP1
Temperature of PP0
DIMM
Temperature of PP0
DIMM
Temperature of PP0
DIMM
Temperature of PP0
DIMM
Temperature of PP1
DIMM
Temperature of PP1
DIMM
Temperature of PP1
DIMM
Temperature of PP1
DIMM
Temperature Inlet
ADT7461
ADT7461
LM75
Temperature Outlet LM75
Temperature P220 cpu MAX1617
ATCA-9405 Installation and Use (6806800M71F) 129
Intelligent Peripheral Management Controller
Table 8-4 Voltage and Temperature Sensor Devices (continued)
I2C address I2C bus
Int ADC MMC
0x94 MMC
Domain
RTM
Mezzanine
Card
Purpose
12V
3.3V
1.8V
0.9V
Temperature
Device
ATMEL ADC
LM75
8.6
POST
POST is executed at IPMC startup when either a hard (blade physically extracted/reinserted) or cold (IPMI Command) reset is performed. POST verifies the functionality of SRAM, IPMB-0,
EEPROM data storage, FRU-Information, and all devices (primarily sensors) attached to the
IPMCs private master-only I2C bus. A detailed description of POST tests are as follows:
130
FRU-Information - This test verifies that the FRU-Information is readable from the external
EEPROM where it is stored. Once read, each section's checksum is computed and validated.
IPMB-0 - This test reads the ready signals coming from the I2C buffers. This test passes as long as both ready signals are active and both IPMB busses (IPMB-A and IPMB-B) are enabled.
EEPROM - This test verifies that the EEPROM contents are readable via I2C. Since the IPMC stores its runtime and persistent data here, proper operation is crucial.
Master-Only I2C - This test verifies that all expected devices attached to the master-only
I2C bus are accessible.
The IPMC contains a sensor of type 0x28 (Management Subsystem Health) which reports the results of the IPMCs POST. If all tests pass, then the sensor reads Performance Met; otherwise it reports Performance Lags. If POST fails, the POST sensor generates an event to the SEL with the Performance Met/Performance Lag offsets.
To obtain results of POST, the IPMC supports the IPMI standard command Get Self Test
Results
with OEM extensions. This IPMI command can be run at anytime.
ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.7
FRU Inventory
The ATCA-9405 implements two intelligent FRU's (IPMC and MMC).
Every FRU provides its own FRU information (serial, part, MAC addresses). Depending on the presence of a module, its FRU information is visible or not.
Table 8-5 FRU Information and SEL at EEPROM Storage
I2C Address
0xA2
0xA6
Device internal
I2C bus
IPMC
IPMC
MMC
Domain
Front blade
Front blade
RTM
Purpose
FRU Information
SEL
FRU Information
The FRU of the RTM is not hot-swappable. This is important to ensure that the system management application (HPI-B) does not have to deal with dynamic FRU population.
Dynamic inventory data of the SFPs is read at OS level.
The MAC addresses of an FRU are stored within the multi-record area of the FRU information.
Emerson Network Power has defined a MAC address multi-record for this purpose, for more
information see MAC Address FRU OEM records on page 131 .
8.7.1
MAC Address FRU OEM records
The Emerson Network Power MAC Address record is specified in
Table 8-6 Emerson ECC MAC Address Record
Offset
0
1
2
Length
1
1
1
Description
Record Type ID. A value of C0h (OEM) is used for Emerson ECC OEM records.
End of List/Version
[7] End of List. Set to 1b for the last record
[6:4] Reserved. Write as 000b.
[3:0] Record format version. Write as 2h.
Record Length
ATCA-9405 Installation and Use (6806800M71F) 131
Intelligent Peripheral Management Controller
Table 8-6 Emerson ECC MAC Address Record (continued)
8
9
10
11
6
7
4
5
Offset
3
1
1
1
N*9
1
1
1
1
Length
1
Description
Record Checksum (zero checksum)
Header Checksum (zero checksum)
LSB of Manufacturer ID. Write as CDh.
Second Byte of Manufacturer ID. Write as 65h.
MSB of Manufacturer ID. Write as 00h.
Motorola Record ID. 01h for Emerson ECC MAC Address Record.
Record Format Version. 01h for this specification.
Number of MAC Address Descriptors (N).
Emerson ECC MAC Address Descriptors. See
Table 8-7 on page 132 , Emerson
ECC MAC Address Descriptor.
Table 8-7 Emerson ECC MAC Address Descriptor
Offset
0
1
2
3
Length
1
1
1
6
Description
Interface Type. See
Length Identifier (for example: 6 =
48 bit MAC, 8 = WWPN)
MAC Address Count (M) (specifying a continuous pool of MAC addresses starting with the MAC address specified in this descriptor)
M = 1: this descriptor specifies one
MAC address
M > 1: this descriptor specifies a pool of MAC addresses with M count
MAC Address. (Canonical form, the
LSB (least significant bit) first.
132 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-8 Interface Type Assignments
08h
09h
10h
11h
04h
05h
06h
07h
Interface type
01h
02h
03h
11h - FFh
Description
ATCA Base Interface
ATCA Base Interface
Front/Rear Panel
Mezzanine Module
Serial over LAN (SOL)
Fibre Channel / WWPN
AMC/MicroTCA Common Options Region
AMC/MicroTCA Fat Pipe Region
AMC/MicroTCA Extended Fat Pipe Region
ATCA Update Channel
Multi-type (Base, Fabric, and Update channel (or two types of it) are connected to a onboard switch) reserved
The SP provides 12 MAC addresses in its FRU information:
2 x P2020 to the switch
1 x P2020 to the front
1 x Terminal Server
3 x 2 CN6880 to the switch
1 x 2 CN6880 to the front
8.8
Reset and Power Domains
The ATCA-9405 provides the following FRU instances:
FRU #0: front board management and switch
FRU #1: RTM
ATCA-9405 Installation and Use (6806800M71F) 133
Intelligent Peripheral Management Controller
Each FRU instance can be reset separately. Each FRU instance is implemented as a managed
FRU having its own hot-swap state machine and FRU information.
8.9
Power Management
The ATCA-9405 provides three power levels to be selected by the ShMM. The IPMC selects the cavium core frequency of both processors to manage blade power consumption. At Power
Level 1, both processors are configured for 800MHz only, at power level 2, both processors run at 1000MHz, and at power level 3, both processors run at their maximum speed.
The power levels to be provided by the IPMC depend on the cavium cpus version and capabilities. The IPMC is evaluating the FRU information of the board. The IPMC provides just two power levels in case the Cavium cpus equipped support just 1000MHz.
Table 8-9 Power Levels
2
3
Power level
1
Cavium CPU frequency
[MHz]
800
1000
1200
Power consumption
[watts]
200
300
400
8.10 U-Boot Boot Configuration Parameters
The IPMC stores u-boot environment variables, which match the variables saved in the nonvolatile storage of P2020. When u-boot starts, it first copies its environment parameter set from its NVRAM into memory. Then it reads parameters from the IPMC, adds new parameters to the parameter set in memory, and deletes or modifies existing ones. During runtime, only the memory copy of the parameter set is used.
The parameters stored in the IPMC are not automatically saved back to the NVRAM of u-boot.
The IPMI command used to manage the boot configuration variables is Set/Get System
Boot Options
together with parameter #100.
The advantage of storing u-boot environment variables in IPMC non-volatile memory is that those can be set by the ShMM or across HPI applications as well. The system manager decides from which boot device the blade should boot from.
134 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
The boot configuration parameters are stored as sets of <parameter name> and <value> pairs. Thus, they can be easily enhanced and there are no dependencies between different versions of IPMC firmware and payload firmware. The IPMC provides a set of boot configuration parameters and the payload firmware initializes those parameters.
Figure 8-4 IPMC Boot Parameter Storage Configuration Flow
ATCA-9405 Installation and Use (6806800M71F) 135
Intelligent Peripheral Management Controller
the System Boot Options parameter #100.
Table 8-10 IPMC Boot Parameter Storage Format
Byte
0-1
Description
Number of bytes used for boot parameters (LSB first)
2-n
N+1 - n+2
The number of bytes must be calculated and written into these two bytes by the software which writes into the storage area. The values 0x0000 and 0xFFFF indicate that no data has been written to the storage area. If you are reading from the storage area and you find any of these two values, your software assumes that no boot firmware boot options have previously been written to the storage area.
Boot Parameters data
The boot parameters are stored as ASCII text with the following general format: <name>=<value>, where all name/value pairs are separated by a zero byte. The end of the boot parameter data is indicated by two zero bytes. Allowed and supported name/value pairs are blade specific.
16 bit checksum over the boot parameters data section (LSB first)
When writing or reading from the storage area, you can only read or write chunks of 16 bytes at a time. For this reason, the IPMC memory is divided into numbered blocks of 16 bytes which need to be addressed individually. For this purpose the block selector field in the request data field is used.
8.11 Asynchronous Event Notification
To inform payload applications about graceful shutdown/reboot requests, the FRU
Activate (Deactivate) and FRU Control (Graceful Reboot) command message is routed as a LUN2 message to payload interface.
If the payload application has registered to these commands via OpenIPMI library, it gets informed and can take all necessary actions before the payload is gracefully rebooted/shutdown.
136 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Graceful Reboot and Graceful Shutdown is also communicated to the Intel CPU via internal communication channel.
8.12 Serial Line Selection
All payload serial interfaces (COM #0 of PP #0, PP #1, SP) of the ATCA-9405 can be redirected to the terminal server. It is even possible to redirect two serial interfaces to the terminal server in parallel. Thus it is possible to have one terminal server specific channel configured to redirect the SP serial interface and the second channel configured to redirect either the serial interface of the first octeon, or of the second one. It is not possible to have both serial interfaces of the octeon redirected to the terminal server.
Note that the serial instance 0 of the terminal server is reserved for the serial interface of the SP and instance 1 for either PP #0 or PP #1.
There are three connectors at the front panel. One connector (serial instance 0) is used to share the serial interface of the SP and the IPMC and the other two (serial instance 1 and 2) are used to connect to the serial interfaces of the cavium cpus.
In addition, there is a serial connector at the RTM. Either the serial interface of the first octeon or the serial interface of the second octeon is redirected to this interface.
ATCA-9405 Installation and Use (6806800M71F) 137
Intelligent Peripheral Management Controller
There is a combined board serial line selection available. The serial line selection is implemented via OEM IPMI command.
Figure 8-5 IPMC Serial Line Selection
138 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
The serial line selection is implemented non-persistently to ensure that the serial interfaces always can be accessed easily after power-up. By default, the serial line selection selects the front-panel connectors after power-up.
8.13 Built-in Terminal Server
The ATCA-9405 provides a built-in terminal server (TS) based on the embedded controller
MCF5223X ColdFire from Freescale. All serial payload data can be accessed via the Terminal
Server.
The IPMC selects the serial source to be routed to the terminal server and configures the TS specific parameters via a separate communication interface (I2C). The parameters for the network interface of the terminal server are:
MAC address
IP addresses
Net mask
Gateway IP address
Baud rate
Depending on the serial line selection, either the payload serial interface of the SP and the serial interface of the first cavium cpu, or the payload serial interface of the SP and the serial interface of the second cavium cpu can be accessed both at a time via telnet protocol.
The Shelf Manager configures the TS using the IPMI commands Set/Get LAN Configuration
Parameters and Set/Get SOL Configuration parameter.
There are two dedicated TS OEM channels available (channel = 0x05, 0x06) that are defined to be used with both Base Interfaces.
The built-in terminal server is not implemented with the use of RMCP packets. Furthermore there is no SOL client required to establish a telnet session.
ATCA-9405 Installation and Use (6806800M71F) 139
Intelligent Peripheral Management Controller
Due to the fact, that there is no authenticated IPMI communication to the IPMC, the channels are implemented IPMI session-less. For TS channel properties, see
.
Table 8-11 TS Channel Information
Channel Protocol Type
Channel Medium Type
Session support
Vendor ID
Auxiliary Channel Info
OEM Protocol
OEM
Session-less
EMERSON IANA
0x1C
0x60
0x00
0xCD 0x65 0x00
8.13.1 Evaluating the Version of the Telnet Server Firmware
The firmware version of the terminal server can be retrieved with the IPMI command Get SOL
Configuration,
executed with the OEM parameter 192.
ipmicmd -k "0 <ipmb slot> 0 c 22 5 C0 0 0" smi 0
8.13.2 Establishing a Telnet Session
SOL client is not required to establish a serial connection to the ATCA-9405 payload serial line via network Base Interface #1 or #2. The TS can be configured very easily via IPMI commands.
To establish a telnet session at OEM channel 5, follow the configuration steps executed from the ShMM first (steps in bold are mandatory if IP address of TS is not default):
1. This step is not required if LAN parameters are not modified (step 2 to 4 is not executed).
To enable LAN configuration, the Set_In_Progress flag must be set: ipmicmd -k "0 <ipmb slot> 0 c 1 5 0 1" smi 0
2. This step is not required if the IPMC default is used: default IP address 172.17.0.220.
To set the IP address, execute the following command: ipmicmd -k "0 <ipmb slot> 0 c 1 5 3 <ip addr>" smi 0 for example, if the IP addresss is 192.168.1.100, then the command is:
ipmicmd -k "0 <ipmb slot> 0 c 1 5 3 c0 a8 01 64" smi 0
140 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Note: This value is stored internally and sent to the TS when the SOL_ENABLE flag is set with the IPMI command Set SOL Configuration. This IP address is invalidated by clearing the
SOL_ENABLE
flag.
3. This step is not required if the IPMC default is used: default subnet mask 255.255.0.0
To set the subnet mask, execute the following commands: ipmicmd -k "0 <ipmb slot> 0 c 1 5 6 <subnet mask>" smi 0
4. This step is not required if the IPMC default is used: default gateway IP address
172.17.0.254
To set the gateway IP address, execute the following commands: ipmicmd -k "0 <ipmb slot> 0 c 1 5 c <ip addr>" smi 0
5. To enable SOL configuration, the Set_In_Progress flag must be set ipmicmd -k "0 <ipmb slot> 0 c 21 5 0 1" smi 0
6. This step is not required if the IPMC default is used: default baud rate is 9600
To set the non-volatile bit-rate, execute the following command: ipmicmd -k "0 <ipmb slot> 0 c 21 5 5 <bit-rate>" smi 0
Note: bit-rate 6 = 9600baud, 7 =19.2kbaud, 8=38.4kbaud
7. To set the SOL_ENABLE flag, execute the following commands: ipmicmd -k "0 <ipmb slot> 0 c 21 5 1 1 smi 0
Note: Telnet session is not possible without enabling the SOL.
ATCA-9405 Installation and Use (6806800M71F) 141
Intelligent Peripheral Management Controller
8. To redirect the serial line of the SP from front connector to the TS, execute the following command: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 0 4" smi 0
9. Open the telnet session telnet <IP address>
To configure channel 6 of the terminal server, execute command in steps 1-7 again, but replace the channel argument from 5 to 6 and do not use the same IP address. Two telnet sessions are possible at the same time.
To redirect the serial line of the PP #0 from front connector to the TS, execute the following command: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 0 0" smi 0
To redirect the serial line of the PP #1 from front connector to the TS, execute the following command: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 1 1" smi 0
For more details on terminal server, refer
Built-in Terminal Server on page 139 .
8.14 Fail Safe Logic and Watchdog Support
The IPMC firmware supports automatic fail safe logic for the payload firmware on the SP.
8.14.1 SP BMC Watchdog
When the IPMC transitions to M4 (Active), the IPMC automatically enables the BMC Watchdog with the following settings:
Timer Use: BIOS/POST
Timer Actions: Hard Reset
Timer Countdown Value: 30 Seconds
SP firmware (U-Boot) must disable or reset the BMC Watchdog within 30 seconds of payload activation. If there is a failure in disabling or resetting the BMC Watchdog, then the IPMC performs a payload reset.
142 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
The IPMC automatically switches the boot banks on SP, provided the failsafe logic is enabled via
OEM command.
When the fail safe logic is triggered as a result of the BMC Watchdog timeout, a System
Firmware Progress sensor SEL event is logged as follows:
Event Data Byte 1: 0xA1 (System Firmware Hang)
Event Data Byte 2: 0x00 (SPP CPU)
Event Data Byte 3: 0xXX (Failed Boot Bank ID: 0=Bank A; 1=Bank B)
Fail Safe logic makes three attempts to boot the payload successfully. After three attempts, the fail safe logic is automatically disabled and the boot bank is left in the original state (before the payload was booted). In addition, this logic is only enabled upon a hard reset of the IPMC firmware, a cold or warm IPMC reset does not enable this functionality.
Fail Safe is disabled by default and can be enabled with the IPMI command Set Feature
Configuration
. For more information, see Set Feature Configuration Command on page 149 .
8.15 Payload Interface
The IPMC communicates with the payload via its host Keyboard Style Controller (KCS) interface. The Renesas H8S provides support for LPC/KCS in hardware. KCS is defined by the
IPMI 1.5 Specification.
8.16 Payload Boot Bank Selection
The ATCA-9405 provides redundant payload boot flashes for crisis recovery. The IPMC manages from which boot bank the payload should boot from.
ATCA-9405 Installation and Use (6806800M71F) 143
Intelligent Peripheral Management Controller
The IPMI command Set System Boot Options together with the parameter #96 is used to specify the payload boot-bank from which the payload should boot from.
Figure 8-6 Payload Boot Bank Selection
8.17 Settable Graceful Shutdown Timeout
The IPMI command Set System Boot Options together with the parameter #98 is used to specify the timeout for Graceful Shutdown. The value of the graceful shutdown timeout is specified for both CPUs.
By default, this value is set to 10 sec.
144 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.18 FPGA Health Check
During normal operation the health of the FPGA is periodically checked by dedicated FPGA internal logic. In the case of a CRC error, the signal CONF_CRC_ERR is asserted to inform the system that FPGA logic may be corrupted. Such an error is handled like a power failure.
The IPMC provides a dedicated FPGA Status sensor indicating a CONF_CRC_ERROR. The ShMM is informed about CONF_CRC_ERROR with an event. In such a case the blade is shutdown by the IPMC.
8.19 Local System Event Log (SEL)
The ATCA-9405 IPMC supports a local SEL. The local SEL size is configured to hold 1K entries in a circular FIFO buffer. Once the circular buffer is full, the next SEL entry overwrites the oldest SEL entry in the buffer. All events are automatically logged locally to the local SEL, before being passed to the SEL of the Shelf. This includes all events that occur from the local MMC.
To support the local SEL, a software emulated RTC (Real Time Clock) is enabled which upon startup requests the local time from the shelf manager by sending an IPMI standard command
Get SEL Time
. Once the initial time is received, the IPMC maintains the time locally and no further synchronization is performed with the shelf manager.
8.20 IPMI Hardware Watchdog
For crisis recovery purpose, the IPMI building block provides a hardware watchdog. The IPMI firmware is reset, if it does not trigger the watchdog.
ATCA-9405 Installation and Use (6806800M71F) 145
Intelligent Peripheral Management Controller
8.21 Emerson OEM Command Set
In addition to standard commands defined by IPMI and PICMG specifications, Emerson defines a set of OEM commands to extend the features that Emerson products may have. Many features are product-specific; therefore, not all OEM commands are implemented on a product. Refer to the document of the particular product for the complete command set implemented on the product.
Table 8-12 Emerson OEM Commands
Command
Set Serial Output
Get Serial Output
Set Feature Configuration
Get Feature Configuration
CMD
15h
16h
1Eh
1Fh
Defined in
2.20.1
0
2.20.3
2.20.4
Emerson OEM request messages uses NetFn 2Eh and the response messages uses NetFn 2Fh.
8.21.1 Set Serial Output Command
This command allows you to set the serial output source for a particular serial port connector.
Table 8-13 Set Serial Output Command
Byte
Request
Data
1
2
3
Data field
LSB of Emerson IANA Enterprise Number. A value of
CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of
00h is used.
146 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-13 Set Serial Output Command (continued)
Byte
4
Response
Data
5
6
1
2
3
4
Data field
Serial connector type:
0 = Front panel connector
1 = RTM panel connector
2 = reserved
3 = Onboard device (that is, Terminal Server)
All other values are reserved.
Serial connector instance number. A sequential number starts from zero.
Serial output selector. For more information, see
Completion Code
LSB of Emerson IANA Enterprise Number. A value of
CDh is used
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of
00h is used.
The following serial output selector assignments can be mapped to a serial connector type and instance:
Table 8-14 Serial Output Selector Assignments
Serial Output Source
Payload serial interface PP #0
Payload serial interface PP #1
IPMC serial interface 1 reserved
Payload serial interface SP reserved
Serial Output Selector
00h
01h
02h
03h
04h
05h - ffh
ATCA-9405 Installation and Use (6806800M71F) 147
Intelligent Peripheral Management Controller
IPMI command examples:
To set the serial COM #0 of SP to the front connector instance 0 (default): ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 0 0 4" smi 0
To set the IPMC serial to front connector instance 0: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 0 0 2" smi 0
To set the serial COM #0 of PP #0 to the front connector instance 1 (default): ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 0 1 0" smi 0
To set the serial COM #0 of PP #1 to the front connector instance 2 (default): ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 0 2 1" smi 0
To set the serial COM #0 of SP to the RTM connector instance 0: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 1 0 2" smi 0
To set the serial COM #0 of SP to the terminal server # 0: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 0 4" smi 0
To set the serial COM #0 of SP to the terminal server #1: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 1 4" smi 0
To set the serial COM #0 of PP #0 to the terminal server #0: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 0 0" smi 0
To set the serial COM #0 of PP #1 to the terminal server #1: ipmicmd -k "0 <ipmb slot> 0 2e 15 cd 65 0 3 1 1" smi 0
8.21.2 Get Serial Output Command
This command allows you to determine which serial output source goes to a particular serial port connector.
Table 8-15 Get Serial Output Command
Byte
Request
Data
1
2
3
Data field
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
148 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
Table 8-15 Get Serial Output Command (continued)
Byte
4
Response
Data
5
1
2
3
4
5
Data field
Serial connector type:
0 = Front panel connector
1 = RTM panel connector
2 = reserved
3 = On-board device (that is, Terminal Server, P4080 COM #1 to Intel
COM #0)
All other values are reserved.
Serial connector instance number. A sequential number starts from zero.
Completion Code
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
Serial output selector. For more information, see
.
8.21.3 Set Feature Configuration
This command is used to enable/disable features within the IPMC during runtime.
Table 8-16 Set Feature Configuration Command
Byte
Request
Data
1
2
3
4
Data field
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
Feature Selector. For more information, see
ATCA-9405 Installation and Use (6806800M71F) 149
Intelligent Peripheral Management Controller
Table 8-16 Set Feature Configuration Command (continued)
Byte
5
6
Response
Data
1
2
3
4
Data field
Feature Configuration.
00h = disabled (Feature Selector = E0)
01h = enabled (Feature Selector = E0)
02h = restore factory default (golden) (Feature Selector = E1)
C0h = reload selected FPGA image (Feature Selector = E1)
03h - FFh = reserved
Persistency / Duration
00h = volatile. Actual duration depends on implementation.
01h - FFh = reserved
Completion Code is Generic, plus the following command-specific completion codes:
80h = feature selector not supported.
81h = feature configuration not supported
82h = configuration persistency / duration not supported
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
Table 8-17 on page 150 , provides the feature set supported with ATCA-9405:
Table 8-17 Feature Selector Assignments
Feature Selector
E0h
E1h
Description
FAILSAFE Function
Enable/Disable
Select FPGA flash
150 ATCA-9405 Installation and Use (6806800M71F)
Intelligent Peripheral Management Controller
8.21.4 Get Feature Configuration
This command is used to retrieve the IPMI feature set being configured.
Table 8-18 Get Feature Configuration Command
Byte
Request
Data
1
2
3
4
Response
Data
1
4
5
2
3
6
Data field
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
Feature Selector. For more information, see
Completion Code is Generic, plus the following command-specific completion codes:
80h = feature selector not supported.
LSB of Emerson IANA Enterprise Number. A value of CDh is used.
2nd byte of Emerson IANA Enterprise Number. A value of 65h is used.
MSB of Emerson IANA Enterprise Number. A value of 00h is used.
Feature Configuration
00h = disabled (Feature Selector = E0)
01h = enabled (Feature Selector = E0)
02h = restore factory default (golden) (Feature Selector = E1)
C0h = reload selected FPGA image (Feature Selector = E1)
C1h = enabled and activated (Feature Selector = E0)
03h - FFh = reserved
Persistency / Duration
ATCA-9405 Installation and Use (6806800M71F) 151
Intelligent Peripheral Management Controller
152 ATCA-9405 Installation and Use (6806800M71F)
Appendix A
A
Related Documentation
A.1
Emerson Network Power - Embedded
Computing Documents
The publications listed below are referenced in this manual. You can obtain electronic copies of
Emerson Network Power - Embedded Computing publications by contacting your local
Emerson sales office. For released products, you can also visit our Web site for the latest copies of our product documentation.
1. Go to www.Emerson.com/EmbeddedComputing . The Emerson Embedded Computing website opens.
2. Click on Technical Documentation link.
3. Click on Search Our Technical Documentation Archive link.
4. In the Search box, type the publication number of the manual you are looking for.
Table A-1 Emerson Network Power - Embedded Computing Publications
Document Title
ARTM-9405 Installation and Use
ATCA-9405 Quick Start Guide
Publication Number
6806800N04
6806800N10
ATCA-9405 Installation and Use (6806800M71F) 153
Related Documentation
154 ATCA-9405 Installation and Use (6806800M71F)
Safety Notes
This section provides warnings that precede potentially dangerous procedures throughout this manual. Instructions contained in the warnings must be followed during all phases of operation, service, and repair of this equipment. You should also employ all other safety precautions necessary for the operation of the equipment in your operating environment.
Failure to comply with these precautions or with specific warnings elsewhere in this manual could result in personal injury or damage to the equipment.
Emerson intends to provide all necessary information to install and handle the product in this manual. Because of the complexity of this product and its various uses, we do not guarantee that the given information is complete. If you need additional information, ask your Emerson representative.
The product has been designed to meet the standard industrial safety requirements. It must not be used except in its specific area of office telecommunication industry and industrial control.
Only personnel trained by Emerson or persons qualified in electronics or electrical engineering are authorized to install, remove or maintain the product.
The information given in this manual is meant to complete the knowledge of a specialist and must not be used as replacement for qualified personnel.
Keep away from live circuits inside the equipment. Operating personnel must not remove equipment covers. Only factory authorized service personnel or other qualified service personnel may remove equipment covers for internal subassembly or component replacement or any internal adjustment.
Do not install substitute parts or perform any unauthorized modification of the equipment or the warranty may be voided. Contact your local Emerson representative for service and repair to make sure that all safety features are maintained.
EMC
This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications.
ATCA-9405 Installation and Use (6806800M71F) 155
Safety Notes
Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense. Changes or modifications not expressly approved by Emerson could void the user's authority to operate the equipment. Board products are tested in a representative system to show compliance with the above mentioned requirements. A proper installation in a compliant system will maintain the required performance. Use only shielded cables when connecting peripherals to assure that appropriate radio frequency emissions compliance is maintained.
Installation
Before installing the board make sure the requirements listed in section "Board Exchange" are met.
Restricted access area - This board is only to be installed in a restricted access area.
Removing the board with the blue LED still blinking causes data loss.
Wait until the blue LED is permanently illuminated, before removing the board.
Damage of Circuits
Electrostatic discharge and incorrect board installation and removal can damage circuits or shorten their life.
Before touching the board or electronic components, make sure that you are working in an
ESD-safe environment or wear ESD wrist straps.
Incorrect board installation and removal can result in board malfunctioning.
Make sure that the board is connected to the system backplane via all assembled connectors and that power is available on all zone 1 power pins.
Damage of the Product
Incorrect installation of the product can cause damage of the product,
Only use handles when installing/removing the product to avoid damage/deformation to the face plate and/or PCB.
156 ATCA-9405 Installation and Use (6806800M71F)
Safety Notes
Damage of the Product and Additional Devices and Modules
Incorrect installation or removal of additional devices or modules may damage the product or the additional devices or modules.
Before installing or removing additional devices or modules, read the respective documentation.
Operation
Board surface
High humidity and condensation on the board surface causes short circuits.
Do not operate the board outside the specified environmental limits. Make sure the board is completely dry and there is no moisture on any surface before applying power.
Board Overheating and Board Damage
Operating the board without forced air cooling may lead to board overheating and thus board damage.
When operating the board, make sure that forced air cooling is available in the shelf.
Injuries or Short Circuits
Board or power supply
In case the ORing diodes of the board fail, the board may trigger a short circuit between input line A and input line B so that line A remains powered even if it is disconnected from the power supply circuit (and vice versa).
To avoid damage or injuries, always check that there is no more voltage on the line that has been disconnected before continuing your work.
Hot Swap
Installing the board into or removing it from a powered system not supporting hot swap or high availability causes board damage and data loss. Therefore, only install it in or remove it from a powered system if the system itself supports hot swap or high availability and if the system documentation explicitly includes guidelines.
ATCA-9405 Installation and Use (6806800M71F) 157
Safety Notes
RJ-45 Connectors
The RJ-45 connectors on the face plate must only be used for twisted-pair Ethernet (TPE) and serial console connections (according to face plate marking). Connecting a telephone to such a connector may destroy your telephone as well as your board. Therefore:
Clearly mark TPE connectors near your working area as network connectors.
Only connect TPE bushing of the system to safety extra low voltage (SELV) circuits.
Make sure that the length of the electric cable connected to a TPE bushing does not exceed 100 m.
If you have further questions, ask your system administrator.
Replacement/Expansion
Only replace or expand components or system parts with those recommended by Emerson.
Otherwise, you are fully responsible for the impact on EMC or any possible malfunction of the product.
Check the total power consumption of all components installed (see the technical specification of the respective components). Ensure that any individual output current of any source stays within its acceptable limits (see the technical specification of the respective source).
Battery
Blade Damage
Wrong battery installation may result in hazardous explosion and blade damage. Therefore, always use the same type of Lithium battery as is installed and make sure the battery is installed as described in this manual.
Environment
Always dispose of used products according to your country’s legislation and manufacturer’s instructions.
158 ATCA-9405 Installation and Use (6806800M71F)
Sicherheitshinweise
Dieses Kapitel enthält Hinweise, die potentiell gefährlichen Prozeduren innerhalb dieses
Handbuchs vorangestellt sind. Beachten Sie unbedingt in allen Phasen des Betriebs, der
Wartung und der Reparatur des Systems die Anweisungen, die diesen Hinweisen enthalten sind. Sie sollten außerdem alle anderen Vorsichtsmaßnahmen treffen, die für den Betrieb des
Produktes innerhalb Ihrer Betriebsumgebung notwendig sind. Wenn Sie diese
Vorsichtsmaßnahmen oder Sicherheitshinweise, die an anderer Stelle dieses Handbuchs enthalten sind, nicht beachten, kann das Verletzungen oder Schäden am Produkt zur Folge haben.
Emerson ist darauf bedacht, alle notwendigen Informationen zum Einbau und zum Umgang mit dem Produkt in diesem Handbuch bereit zu stellen. Da es sich jedoch um ein komplexes
Produkt mit vielfältigen Einsatzmöglichkeiten handelt, können wir die Vollständigkeit der im
Handbuch enthaltenen Informationen nicht garantieren. Falls Sie weitere Informationen benötigen sollten, wenden Sie sich bitte an die für Sie zuständige Geschäftsstelle von Emerson.
Das System erfüllt die für die Industrie geforderten Sicherheitsvorschriften und darf ausschließlich für Anwendungen in der Telekommunikationsindustrie und im Zusammenhang mit Industriesteuerungen verwendet werden.
Einbau, Wartung und Betrieb dürfen nur von durch Emerson ausgebildetem oder im Bereich
Elektronik oder Elektrotechnik qualifiziertem Personal durchgeführt werden. Die in diesem
Handbuch enthaltenen Informationen dienen ausschließlich dazu, das Wissen von
Fachpersonal zu ergänzen, können dieses jedoch nicht ersetzen.
Halten Sie sich von stromführenden Leitungen innerhalb des Produktes fern. Entfernen Sie auf keinen Fall Abdeckungen am Produkt. Nur werksseitig zugelassenes Wartungspersonal oder anderweitig qualifiziertes Wartungspersonal darf Abdeckungen entfernen, um Komponenten zu ersetzen oder andere Anpassungen vorzunehmen.
Installieren Sie keine Ersatzteile oder führen Sie keine unerlaubten Veränderungen am Produkt durch, sonst verfällt die Garantie. Wenden Sie sich für Wartung oder Reparatur bitte an die für
Sie zuständige Geschäftsstelle von Emerson. So stellen Sie sicher, dass alle sicherheitsrelevanten Aspekte beachtet werden.
ATCA-9405 Installation and Use (6806800M71F) 159
Sicherheitshinweise
EMV
Das Produkt wurde in einem Emerson Standardsystem getestet. Es erfüllt die für digitale
Geräte der Klasse A gültigen Grenzwerte in einem solchen System gemäß den FCC-Richtlinien
Abschnitt 15 bzw. EN 55022 Klasse A. Diese Grenzwerte sollen einen angemessenen Schutz vor Störstrahlung beim Betrieb des Produktes in Gewerbe- sowie Industriegebieten gewährleisten.
Das Produkt arbeitet im Hochfrequenzbereich und erzeugt Störstrahlung. Bei unsachgemäßem Einbau und anderem als in diesem Handbuch beschriebenen Betrieb können
Störungen im Hochfrequenzbereich auftreten.
Wird das Produkt in einem Wohngebiet betrieben, so kann dies mit großer Wahrscheinlichkeit zu starken Störungen führen, welche dann auf Kosten des Produktanwenders beseitigt werden müssen. Änderungen oder Modifikationen am Produkt, welche ohne ausdrückliche
Genehmigung von Emerson durchgeführt werden, können dazu führen, dass der Anwender die Genehmigung zum Betrieb des Produktes verliert. Boardprodukte werden in einem repräsentativen System getestet, um zu zeigen, dass das Board den oben aufgeführten EMV-
Richtlinien entspricht. Eine ordnungsgemäße Installation in einem System, welches die EMV-
Richtlinien erfüllt, stellt sicher, dass das Produkt gemäß den EMV-Richtlinien betrieben wird.
Verwenden Sie nur abgeschirmte Kabel zum Anschluss von Zusatzmodulen. So ist sichergestellt, dass sich die Aussendung von Hochfrequenzstrahlung im Rahmen der erlaubten
Grenzwerte bewegt.
Warnung! Dies ist eine Einrichtung der Klasse A. Diese Einrichtung kann im Wohnbereich
Funkstörungen verursachen. In diesem Fall kann vom Betreiber verlangt werden, angemessene Maßnahmen durchzuführen.
Board Installation
Bevor Sie das Board in einem System installieren, überprüfen Sie, ob die im Kapitel "Board
Exchange" aufgeführten Anforderungen erfüllt werden.
Bereich mit eingeschränktem Zugang - Installieren Sie das Board in ein System nur in Bereichen mit eingeschränktem Zugang.
160 ATCA-9405 Installation and Use (6806800M71F)
Sicherheitshinweise
Datenverlust
Ziehen Sie das Board im laufenden Betrieb heraus, obwohl die Hot-Swap LED noch nicht leuchtet, führt das zu Datenverlust.
Warten Sie deshalb bis die Hot-Swap LED blau leuchtet, bevor Sie das Board herausziehen.
Beschädigung von Schaltkreisen
Elektrostatische Entladung und unsachgemäßer Ein- und Ausbau des Produktes kann
Schaltkreise beschädigen oder ihre Lebensdauer verkürzen.
Bevor Sie das Produkt oder elektronische Komponenten berühren, vergewissern Sie sich, dass
Sie in einem ESD-geschützten Bereich arbeiten.
Fehlfunktion des Produktes
Fehlerhafter Ein- und Ausbau des Produktes kann zur Beschädigung des Produktes führen.
Stellen Sie deshalb sicher, dass das Produkt mit allen Steckern mit der Systembackplane verbunden ist und über alle Zone-1-Anschlüsse mit Spannung versorgt wird.
Beschädigung des Produktes
Fehlerhafte Installation des Produktes kann zu einer Beschädigung des Produktes führen.
Verwenden Sie die Handles, um das Produkt zu installieren/deinstallieren. Auf diese Weise vermeiden Sie, dass das Face Plate oder die Platine deformiert oder zerstört wird.
Beschädigung des Produktes und von Zusatzmodulen
Fehlerhafte Installation von Zusatzmodulen, kann zur Beschädigung des Produktes und der
Zusatzmodule führen.
Lesen Sie daher vor der Installation von Zusatzmodulen die zugehörige Dokumentation.
Betrieb
Beschädigung des Boards
Hohe Luftfeuchtigkeit und Kondensat auf der Oberfläche des Boards können zu Kurzschlüssen führen.
Betreiben Sie das Board nur innerhalb der angegebenen Grenzwerte für die relative
Luftfeuchtigkeit und Temperatur. Stellen Sie vor dem Einschalten des Stroms sicher, dass sich auf dem Board kein Kondensat befindet.
ATCA-9405 Installation and Use (6806800M71F) 161
Sicherheitshinweise
Überhitzung und Beschädigung des Boards
Betreiben Sie das Board ohne Zwangsbelüftung, kann das Board überhitzt und schließlich beschädigt werden.
Bevor Sie das Board betreiben, müssen Sie sicher stellen, dass das Shelf über eine
Zwangskühlung verfügt.
Verletzungen oder Kurzschlüsse
Board oder Stromversorgung
Falls die ORing Dioden des Boards durchbrennen, kann das Board einen Kurzschluss zwischen den Eingangsleitungen A und B verursachen. In diesem Fall ist Leitung A immer noch unter
Spannung, auch wenn sie vom Versorgungskreislauf getrennt ist (und umgekehrt).
Prüfen Sie deshalb immer, ob die Leitung spannungsfrei ist, bevor Sie Ihre Arbeit fortsetzen, um Schäden oder Verletzungen zu vermeiden.
Hot Swap
Wenn Sie das Board im laufenden Betrieb in ein System, das weder Hot Swap noch High
Availability unterstützt, installieren bzw. herausziehen, wird das Board beschädigt und es gehen Daten verloren. Installieren/entfernen Sie das Board nur im laufenden Betrieb, wenn das
System Hot Swap oder High-Availability unterstützt und wenn die Systembeschreibung dies ausdrücklich erlaubt.
RJ-45 Stecker
Die RJ-45 Stecker auf der Frontblende dürfen nur für Twisted-Pair-Ethernet (TPE) oder für
Serielle Konsole Verbindungen verwendet werden (entsprechend der Markierung an der
Frontblende). Beachten Sie, dass ein versehentliches Anschließen einer Telefonleitung an einen solchen TPE Stecker sowohl das Telefon als auch das Board zerstören kann. Beachten Sie deshalb die folgenden Hinweise:
Kennzeichnen Sie TPE-Anschlüsse in der Nähe Ihres Arbeitsplatzes deutlich als
Netzwerkanschlüsse.
Schließen Sie an TPE-Buchsen ausschließlich SELV-Kreise
(Sicherheitskleinspannungsstromkreise) an.
Die Länge des mit dem Board verbundenen Twisted-Pair Ethernet-Kabels darf 100 m nicht
überschreiten.
162 ATCA-9405 Installation and Use (6806800M71F)
Sicherheitshinweise
Falls Sie Fragen haben, wenden Sie sich bitte an Ihren Systemadministrator.
Austausch/Erweiterung
Verwenden Sie bei Austausch oder Erweiterung nur von Emerson empfohlene Komponenten und Systemteile. Andernfalls sind Sie für mögliche Auswirkungen auf EMV oder Fehlfunktionen des Produktes voll verantwortlich.
Überprüfen Sie die gesamte aufgenomme Leistung aller eingebauten Komponenten (siehe die technischen Daten der entsprechenden Komponente). Stellen Sie sicher, dass die
Stromaufnahme jedes Verbrauchers innerhalb der zulässigen Grenzwerte liegt (siehe die technischen Daten des entsprechenden Verbrauchers).
Batterie
Beschädigung des Blades
Ein unsachgemäßer Einbau der Batterie kann gefährliche Explosionen und Beschädigungen des Blades zur Folge haben.
Verwenden Sie deshalb nur den Batterietyp, der auch bereits eingesetzt wurde und befolgen
Sie die Installationsanleitung.
Umweltschutz
Entsorgen Sie alte Batterien und/oder Produkte stets gemäß der in Ihrem Land gültigen
Gesetzgebung und den Empfehlungen des Herstellers.
ATCA-9405 Installation and Use (6806800M71F) 163
Sicherheitshinweise
164 ATCA-9405 Installation and Use (6806800M71F)
HOW TO REACH LITERATURE AND TECHNICAL SUPPORT:
For literature, training, and technical assistance and support programs, visit www.Emerson.com/EmbeddedComputing
Emerson Network Power.
The global leader in enabling Business-Critical Continuity ™
AC Power Systems
Connectivity
DC Power Systems
Embedded Computing
Embedded Power
Integrated Cabinet Solutions www.Emerson.com/EmbeddedComputing
Outside Plant
Power Switching & Control
Precision Cooling
Services
Site Monitoring
Surge & Signal Protection
Emerson, Business-Critical Continuity, Emerson Network Power and the Emerson Network Power logo are trademarks and service marks of Emerson Electric Co.
All other product or service names are the property of their respective owners.
© 2013 Emerson Electric Co.
advertisement
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Related manuals
advertisement
Table of contents
- 1 ATCA-9405
- 3 Regulatory Agency Warnings & Notices
- 5 Contents
- 15 About this Manual
- 15 Overview of Contents
- 16 Abbreviations
- 18 Conventions
- 19 Summary of Changes
- 21 Introduction
- 21 1.1 Overview
- 22 1.2 Components and Features
- 23 1.3 Functional Overview
- 24 1.4 Additional Information
- 24 1.4.1 Regulatory Compliance
- 26 1.4.2 RoHS Compliance
- 26 1.4.3 Notation
- 26 1.5 Ordering Information
- 27 1.6 Product Identification
- 29 Setup
- 29 2.1 Overview
- 29 2.2 Electrostatic Discharge
- 30 2.3 ATCA-9405 Circuit Board
- 31 2.3.1 Switch Settings
- 33 2.3.1.1 FPGA and CPLD/IPMC Switches
- 36 2.3.2 Safety Critical Hot Spots
- 37 2.3.3 Connector Pin Assignment
- 37 2.3.3.1 Face Plate Connectors
- 39 2.3.3.2 On-board Connectors
- 41 2.3.3.3 Back Panel Connectors
- 46 2.3.4 Debugging Headers
- 47 2.3.4.1 IPMC Debug Console Header
- 47 2.3.4.2 COP Header
- 48 2.3.4.3 EJTAG Header
- 49 2.4 ATCA-9405 Setup
- 50 2.4.1 Power Requirements
- 51 2.4.2 Environmental Considerations
- 53 2.4.3 Hot Swap
- 54 2.5 Troubleshooting
- 54 2.5.1 Technical Support
- 55 2.5.2 Product Repair
- 57 Packet Processor
- 57 3.1 Overview
- 58 3.2 CN6880 Processor
- 58 3.3 Cache
- 59 3.4 System Memory
- 59 3.4.1 Memory Interface
- 59 3.4.2 Memory Socket
- 60 3.4.3 Memory Modules
- 61 3.4.4 Thermal Sensor
- 61 3.5 Octeon U-Boot
- 61 3.5.1 NVRAM
- 62 3.5.2 Network Interfaces
- 62 3.6 SerDes Configuration
- 63 3.7 PCI Express Interface
- 63 3.8 Ethernet Interface
- 64 3.8.1 Front Panel Interface
- 64 3.8.2 Base and Fabric Interface
- 64 3.9 Interlaken Interface
- 65 3.10 USB Interface
- 65 3.11 UART Interface
- 66 3.12 I2C Interface
- 66 3.13 JTAG Interface
- 66 3.14 Interrupts
- 66 3.14.1 Packet Processor Interrupts
- 67 3.15 Power Supply
- 68 3.16 Cooling
- 69 Service Processor
- 69 4.1 Overview
- 70 4.2 P2020 Processor
- 70 4.3 Cache
- 70 4.4 Main Memory
- 70 4.4.1 Memory Interface
- 71 4.4.2 Memory Socket
- 71 4.4.3 Memory Modules
- 72 4.4.4 Persistent Memory
- 73 4.4.5 Thermal Sensor
- 73 4.5 SP U-Boot
- 73 4.5.1 Environment Variables
- 74 4.5.2 Passing Parameter Set to the Operating System
- 74 4.5.3 Dynamic Variables Set During the Boot Phase
- 75 4.5.4 Variables for Controlling the Boot Progress
- 76 4.5.5 Firmware Update
- 76 4.5.6 Application/OS Boot
- 77 4.5.6.1 Default Boot Sequences
- 77 4.5.7 Memory/Address Map Initialization
- 77 4.5.7.1 Address Map
- 78 4.6 Local Bus
- 78 4.7 SerDes Configuration
- 79 4.8 PCI Express Interface
- 80 4.9 Ethernet Interface
- 80 4.9.1 Front Panel Interface
- 80 4.9.2 Base and Fabric Interface
- 81 4.10 SPI Interface
- 81 4.10.1 Boot Flash
- 81 4.10.2 Boot Flash Selection
- 82 4.11 USB Interface
- 82 4.11.1 USB Connector
- 83 4.11.2 e-USB Flash Drive
- 83 4.12 UART Interface
- 83 4.13 I2C Interface
- 84 4.13.1 Real Time Clock (RTC)
- 84 4.14 JTAG Interface
- 84 4.15 Interrupts
- 85 4.15.1 Service Processor Interrupts
- 85 4.16 Cooling
- 87 Ethernet Infrastructure
- 87 5.1 Overview
- 89 5.2 Ethernet Switch
- 90 5.2.1 Port Configuration
- 92 5.2.2 Two-Wire Serial Interface
- 92 5.2.3 Switch Management Interface
- 93 5.2.4 PHY Management Interface
- 94 5.3 Base Interface
- 94 5.4 Fabric Interface
- 97 5.5 Update Channel
- 97 5.6 Serial Redirection
- 99 Service Infrastructure
- 99 6.1 Overview
- 99 6.2 Port Configuration
- 100 6.3 Hot Plug Support
- 101 6.3.1 Serial Hot Plug Controller
- 102 6.4 I2C Slave Interface
- 102 6.5 JTAG Support
- 102 6.6 Lane Status
- 105 Mezzanine Module
- 105 7.1 Overview
- 109 Intelligent Peripheral Management Controller
- 109 8.1 Overview
- 109 8.2 Functional Overview
- 112 8.3 Firmware Architecture
- 113 8.4 HPM.1 Components
- 115 8.4.1 FPGA Firmware Upgrade
- 116 8.4.2 Payload Firmware Upgrade
- 116 8.4.3 IPMC Firmware Upgrade
- 117 8.4.4 Manual Rollback
- 117 8.4.5 Retrieving Versioning Information
- 118 8.5 Sensors
- 126 8.5.1 Firmware Progress, OS Boot, and Boot Error Sensor
- 126 8.5.2 Boot Bank Supervision Sensor
- 127 8.5.3 POST Results Sensor
- 127 8.5.4 Power Good Sensor
- 127 8.5.5 Power Interface Sensors
- 128 8.5.6 Reset Cause Sensor
- 128 8.5.7 Presence Sensors
- 128 8.5.8 Voltage and Temperature Sensors
- 130 8.6 POST
- 131 8.7 FRU Inventory
- 131 8.7.1 MAC Address FRU OEM records
- 133 8.8 Reset and Power Domains
- 134 8.9 Power Management
- 134 8.10 U-Boot Boot Configuration Parameters
- 136 8.11 Asynchronous Event Notification
- 137 8.12 Serial Line Selection
- 139 8.13 Built-in Terminal Server
- 140 8.13.1 Evaluating the Version of the Telnet Server Firmware
- 140 8.13.2 Establishing a Telnet Session
- 142 8.14 Fail Safe Logic and Watchdog Support
- 142 8.14.1 SP BMC Watchdog
- 143 8.15 Payload Interface
- 143 8.16 Payload Boot Bank Selection
- 144 8.17 Settable Graceful Shutdown Timeout
- 145 8.18 FPGA Health Check
- 145 8.19 Local System Event Log (SEL)
- 145 8.20 IPMI Hardware Watchdog
- 146 8.21 Emerson OEM Command Set
- 146 8.21.1 Set Serial Output Command
- 148 8.21.2 Get Serial Output Command
- 149 8.21.3 Set Feature Configuration
- 151 8.21.4 Get Feature Configuration
- 153 A Related Documentation
- 153 A.1 Emerson Network Power - Embedded Computing Documents
- 155 Safety Notes
- 159 Sicherheitshinweise