Chapter 6

Service Infrastructure

6.1

Overview

The Service Processor (SP) monitors and controls the infrastructure of blade and RTM through high speed PCIe interfaces. Connection between the PCIe root complex and PCIe endpoints is provided through PEX8608, a fully non-blocking, low latency, and low power 8-lane, 8-port

PCI-Express Gen 2 switch.

The PEX8608 provides up to eight configurable PCIe ports with x1or x2 link width (number of lanes per unique link). The port numbers are 0, 1, and 4-9 as defined in

Table 6-1 . Any of the

ports can be designated (or dynamically changed) to be the upstream port, that provides connection towards the PCIe root complex.

Link width auto negotiation, dynamic lane reversal, and polarity reversal during link training process is supported for all ports. Other features of the switch are:





Integrated 2.5 or 5.0 GT/s SerDes speed negotiation per port

Non-blocking crossbar switch architecture











Low packet latency and high performance

PCIe power management

Quality of service, reliability, availability, and serviceability features

Out-of-Band initialization options

JTAG support

6.2

Port Configuration

Port configuration is a part of the PEX8608 initialization process that starts upon exiting from the fundamental reset through PEX_PERST# reset input. The configuration of upstream port, port width, and other hardware modes are initially set by hardware strapping signals.

ATCA-9405 Installation and Use (6806800M71F) 99

Service Infrastructure

The physical layer device (PHY) of the configured ports attempt to bring up the links, which includes link training process, link initialization, and automatic link width negotiation.

The ports that are neither configured nor enabled are invisible to the software.

The following table lists the port configuration:

Table 6-1 PEX8608 Port Configuration

Port

Number

Port 0

Port 1

Port 4

Port 5

Port 6

Port 7

Port 8

Port 9

6

3

7

5

2

4

1

Lane

Number

0 x1 x1 x1 x1 x1 x1 x1

Link

Width x1

Root

Complex x

Destination

Service Processor P2020

Packet Processor 1 - CN6880

Packet Processor 2 - CN6880

Ethernet Switch 98CX8234, Port Group 1

Ethernet Switch 98CX8234, Port Group 2

Ethernet Switch 98CX8234, Port Group 3

Ethernet Switch 98CX8234, Port Group 4

RTM Interface

The SP has the role of PCIe root complex, thus Port 0 is configured as default upstream port.

6.3

Hot Plug Support

ATCA-9405 supports software controlled power-down and power-on sequence for the two PP units including memory, Gigabit Ethernet PHY, TCAM module, and all external interfaces. This is used to reduce power consumption of the blade, when application does not need full computing performance of both PPs.

100 ATCA-9405 Installation and Use (6806800M71F)

Service Infrastructure

The application running on the SP is responsible for the power cycle. The two Packet Processor

(PP) units are connected to the SP through PCIe interface, thus the power-down and power-on sequence is handled like a Hot Plug event for standard PCIe plug-in card, insertion and removal process.

Hot Plug is supported for PCIe Port 1 (PP 1) and PCIe Port 4 (PP 2). The implementation is compliant with PCI Hot-Plug Specification [14] and Standard Hot-Plug Controller and Subsystem

Specification [15].

6.3.1

Serial Hot Plug Controller

The PEX8608 supports insertion and removal for all downstream ports through an I2C based

Serial Hot Plug Controller (SHPC) and an external I/O Expander per port. The SHPC is able to control the ports of the I/O Expander through I2C master interface and retrieve the port status, such as device connect status, power fault, or MRL sensor position.

Figure 6-1 PEX8608 Serial Hot Plug Controller

The application running on the SP is responsible for interaction with the SHPC and the additional logic in the Power CPLD and Glue Logic FPGA. The two I/O Expander are implemented in the Glue Logic FPGA.

ATCA-9405 Installation and Use (6806800M71F) 101

Service Infrastructure

6.4

I2C Slave Interface

PEX8608 includes an I2C slave interface. The interface is a sideband mechanism that allows the device configuration registers to be programmed, independent of the PCIe upstream link.

The I2C slave interface is connected to second I2C interface of P2020. For more information,

see I2C Interface on page 83 .

6.5

JTAG Support

The IEEE 1149.1 compliant JTAG boundary scan interface of PEX8608 is connected to the onboard payload JTAG chain.

6.6

Lane Status

Lane status outputs are provided. Each output is directly connected to a small LED to provide visual indication regarding the PHY of each lane. LEDs are encoded as listed below:

 LED offLane is disabled





LED onLane is enabled, 5.0 GT/s

LED blinkingLane is enabled, 2.5 GT/s

Table 6-2 Lane Mapping

D22

D23

D24

D25

D26

D27

D28

D29

RTM

Ethernet Switch Port Group 3

Ethernet Switch Port Group 1

Packet Processor 1

Ethernet Switch Port Group 4

Ethernet Switch Port Group 2

Packet Processor 1

Service Processor

102 ATCA-9405 Installation and Use (6806800M71F)

Figure 6-2 PEX8608 Lane Status LEDs

Service Infrastructure

ATCA-9405 Installation and Use (6806800M71F) 103

Service Infrastructure

104 ATCA-9405 Installation and Use (6806800M71F)