Texas Instruments | PCI Express (PCIe) Resource Wiki for Keystone Devices (Rev. A) | Application notes | Texas Instruments PCI Express (PCIe) Resource Wiki for Keystone Devices (Rev. A) Application notes

Texas Instruments PCI Express (PCIe) Resource Wiki for Keystone Devices (Rev. A) Application notes
Application Report
SPRAC59A – November 2016 – Revised May 2017
PCI Express (PCIe) FAQ for KeyStone™ Devices
ABSTRACT
This document is a collection of frequently asked questions (FAQs) about Peripheral Component
Interconnect Express (PCIe) on the KeyStone™ family of devices. The References Section provides
hardware and software resources and links.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Contents
I have mounted the KeyStone™ I EVM onto the PCIe adapter card and inserted that to the host PC
PCIe slot, but my EVM is not detected after host boot up. Are there any debug suggestions?...................
I had a setup that worked on a host machine. I just changed a host machine or PC, now my EVM is not
enumerated in the new host. What can I do to resolve this? ..........................................................
How do you set the PCIe BAR configuration? ...........................................................................
How do you configure the PCIe interrupt (MSI or legacy interrupt)? Are there any examples?...................
Which version of FPGA firmware requires IBL for PCIe boot mode? .................................................
How do you build and flash the IBL? ......................................................................................
How do you develop the PCIe RC mode driver? Is there any example? ............................................
How do you build and run the PCIe Linux™ host loader demo? ......................................................
When executing the PCIe Linux™ host loader (with interrupt) demo the host PC hangs-up. How do you
resolve this problem?........................................................................................................
What is the purpose of PCIe inbound and outbound address translation? ..........................................
What is the significance of the BAR0 register configuration in PCIe boot? ..........................................
What is the work flow of the MCSDK PCIe example project? .........................................................
Does the C66x device support the PCIe hot plug?......................................................................
How do I disable active state power management (ASPM)? ..........................................................
How do I limit the PCIe speed to GEN1 only? ...........................................................................
How do I limit the PCIe to one lane only? ................................................................................
How do I check the lane width and speed for the PCIe established link? ...........................................
Can I run an application in DDR3 using PCIe boot? ....................................................................
Can I have multiple cores to access PCIe data space at the same time, but in different locations? .............
What is the PCIe data space limitation? ..................................................................................
How do I test PCIe PHY loopback? .......................................................................................
Resources ....................................................................................................................
2
2
3
3
3
3
3
3
4
4
4
4
4
4
4
5
5
5
5
5
5
6
Trademarks
KeyStone is a trademark of Texas Instruments.
Ubuntu is a registered trademark of Canonical.
Linux is a trademark of Linus Torvalds.
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
PCI Express (PCIe) FAQ for KeyStone™ Devices
Copyright © 2016–2017, Texas Instruments Incorporated
1
I have mounted the KeyStone™ I EVM onto the PCIe adapter card and inserted that to the host PC PCIe slot, but my EVM is
not detected after host boot up. Are there any debug suggestions?
www.ti.com
1
I have mounted the KeyStone™ I EVM onto the PCIe adapter card and inserted that
to the host PC PCIe slot, but my EVM is not detected after host boot up. Are there
any debug suggestions?
There are multiple reasons that the EVM might not be detected. Use the Section 1.1, Section 1.2,
Section 1.3 and Section 1.4 subsections to resolve the detection issue.
1.1
AMC-to-PCIe Adapter
Please check the PCB version of the PCIe adapter card. There was a PC compatibility issue with old
versions of AMC-to-PCIe adapter cards. These issues are fixed in the new versions of the adapter card:
PCB REV: 17-00107-03 and PCA REV: 18-00107-03. Please reference AMC to PCIe Adapter Card for the
adapter information.
1.2
Power Sequencing
The following steps show the power sequencing.
1. PCIe slot on the PC host provides power and reference clock to the PCIe module on the EVM.
2. PCIe boot code on the EVM initializes the C66x PCIe module and waits for the link coming up.
3. PCIe root complex (RC) in the PC host is powered up and a link is established between the PCIe RC
in the host and PCIe end point (EP) in the EVM.
4. PC host enumeration (BIOS) starts to scan the PCIe bus.
5. PCIe end point in the EVM is enumerated and registered in the operating system (OS) of the host PC.
1.3
EVM FPGA Version
Please use this section only for the C6657 EVM. Production EVMs are not detected when inserted in ATX
chassis using the TMDXEVMPCI adapter. For more information, refer to the EVM manufacture’s link.
Workaround: The standard IBL (created for the C6657 workaround) contains PCIe configuration writes
that are required to enable robust operation in an ATX computer chassis. The FPGA version v02 redirects
the C6657 to boot from the IBL even when the DIP switches are programmed for PCIe end-point boot.
These PCIe configuration writes are required in the customer application to enable C6657 operation in an
ATX computer.
The production EVM6657 has the V3 FPGA; it will boot from ROM directly and the programed IBL will not
be loaded to run. Downgrade the FPGA to V2 so it will boot from the IBL. The PCIE workaround in IBL
should work for PCIe enumeration for the ATX PC with SSC.
1.4
Debug PCIe Link Status
Use the LTSSM_STATE field (bits [4:0]) in the DEBUG0 register (0x21801728) to show the status of the
PCIe link. It will not be 0x11 if the link is disconnected. Please see Appendix A.1 of the PCIe user guide
for the names of the LTSSM states corresponding to the encoded values.
2
I had a setup that worked on a host machine. I just changed a host machine or PC,
now my EVM is not enumerated in the new host. What can I do to resolve this?
The KeyStone I (6657, 6670, and 6678) EVM is an AMC form factor card that provides multiple
connectivity options for inter-connection with various systems, primarily for telecommunications. In
addition, the device can be connected to a standard PCIe slot through an AMC-to-PCIe adapter card. The
manufacturer had tested the adapter card with multiple platforms, however, the adapter card is not
designed for complete PCIe compatibility. The most significant compatibility issue is the lack of a PCIe
reset control on the adapter card. The PCIe reset is not propagated to the system-on-chip (SoC); this can
cause enumeration issues or boot-up issues. Please try to test the same card in different PC.
2
PCI Express (PCIe) FAQ for KeyStone™ Devices
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
Copyright © 2016–2017, Texas Instruments Incorporated
How do you set the PCIe BAR configuration?
www.ti.com
3
How do you set the PCIe BAR configuration?
Please see the base address registers (BARs) section of KeyStone Architecture Peripheral Component
Interconnect Express (PCIe). Refer to the PCIe BAR configuration defined in IBL source code for
KeyStone I device.
• Function: iblPCIeWorkaround()
• Path: pdk_c667x_x_x_x\packages\ti\boot\ibl\src\device\c66x\c66xinit.c
4
How do you configure the PCIe interrupt (MSI or legacy interrupt)? Are there any
examples?
The MSI and legacy interrupt example project is available in the latest Processor SDK package.
NOTE: PCIe Linux™ host loader demo codes also have legacy interrupt usage when the EVM is
configured as EP mode. Refer to Section 8 for building and running the Linux host loader
example.
5
Which version of FPGA firmware requires IBL for PCIe boot mode?
Refer to Section 1.3.
For FPGA v2, the program counter (PC) of the DSP will be inside LL2 and is booted from RBL to IBL
(loaded from I2C address 0x51 into LL2) and stay inside LL2 forever and will not jump back to RBL
(0x20b0xxxx). For FPGA v3, it boot directly from RBL, no IBL is used.
NOTE: This is applicable for the C6657 EVM only. All of the FPGA versions of the C6678 EVM
require IBL for PCIe boot mode. Refer to the EVM page for more information.
6
How do you build and flash the IBL?
Processor IBL source code path: pdk_c667x_x_x_x\packages\ti\boot\ibl\src\.
Please refer to the README.txt and build_instructions.txt documents on Processor SDK to build and
update IBL: pdk_c667x_x_x_x\packages\ti\boot\ibl\doc\.
7
How do you develop the PCIe RC mode driver? Is there any example?
TI doesn’t provide a PCIe RC enumeration example or driver source. Normally the RC runs an OS that
performs the enumeration with the following steps. If the C66x device is configured as RC, then:
1. The CFG_SETUP register specifies the target bus, device, and function numbers for the target device
(switch and EP).
2. The CFG_SETUP register attempts to read and write the remote device space in MMR (from offset
0x2000 in PCIe MMR space) to generate the configuration requests.
8
How do you build and run the PCIe Linux™ host loader demo?
The Linux host loader example only works on a Linux PC. Please refer to the
pdk_c667x_x_x_x\packages\ti\boot\examples\pcie\docs\Readme.pdf document for more information.
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
PCI Express (PCIe) FAQ for KeyStone™ Devices
Copyright © 2016–2017, Texas Instruments Incorporated
3
When executing the PCIe Linux™ host loader (with interrupt) demo the host PC hangs-up. How do you resolve this problem?
www.ti.com
9
When executing the PCIe Linux™ host loader (with interrupt) demo the host PC
hangs-up. How do you resolve this problem?
The example project was tested on Ubuntu® 10.04 PC 2.6.32-24-generic kernel. The Linux PC cannot
receive the interrupt generated by DSP, so the user must update the grub settings. View and edit the
/etc/default/grub to add quiet splash irqpoll (see the following list).
• Default setting: GRUB_CMDLINE_LINUX_DEFAULT="quiet"
• Recommended setting: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash irqpoll"
Run update-grub after editing /etc/default/grub, then reboot the PC and run the EDMA interrupt test again.
10
What is the purpose of PCIe inbound and outbound address translation?
Inbound address translation remaps accepted incoming accesses from other PCIe devices to locations
within the memory map of the device. Outbound address translation maps the internal bus address to
PCIe address space; this is accomplished by using outbound address translation logic. For each outbound
read and write request, the address translation module within the PCIe subsystem (PCIESS) can convert
an internal bus address to a PCIe address of memory read and write type. For more information, refer to
PCIe User's Guide.
11
What is the significance of the BAR0 register configuration in PCIe boot?
The BAR0 register is fixed to be mapped to the PCIe application registers region; the BAR0 mask setup
does not affect BAR0 register. If you want to use four configurable BAR registers, please configure BAR1
to BAR4 accordingly.
12
What is the work flow of the MCSDK PCIe example project?
In the Processor SDK PCIe example, two DSP EVMs are used to test the PCIe driver. DSP 1 is
configured as an RC and DSP 2 is configured as EP. Once the PCIe link is established, the following
sequence of actions will happen:
1. DSP 1 sends data to DSP 2, and DSP 2 waits to receive all of the data.
2. DSP 2 sends the data back to DSP 1.
3. DSP 1 waits to receive all of the data.
4. DSP 1 verifies if the received data matches the sent data.
5. DSP 2 sends MSI and legacy interrupts to DSP 1.
13
Does the C66x device support the PCIe hot plug?
The PCIe in C66x device does not support the hot plug feature, and the card presence detect pin PRSNT
is not supported. And PERST as a side-band reset signal is also optional in the PCIe specification and is
not supported in the C66x PCIe. TI makes a family of redriver and repeater parts that offer the hot-plug
capability. A component like the DS80PCI102 is designed specifically for PCIe. This component has a
control signal (PRSNT#) that disables the output drivers that would be connected to the C66xx.
14
How do I disable active state power management (ASPM)?
The ASPM control field (ACTIVE_LINK_PM) in the Link Status and Control Register (LINK_STAT_CTRL)
should be set to 0 to disable ASPM. It is recommended to change the advertised ASPM capabilities
through the AS_LINK_PM field in the Link Capabilities Register (LINK_CAP). By default, AS_LINK_PM is
set to advertise L0s support; changing this field to a value of 0 indicates no support for ASPM. Set
LINK_CAP.AS_LINK_PM = 0 and LINK_STAT_CTRL.ACTIVE_LINK_PM = 0 by calling APIs.
15
How do I limit the PCIe speed to GEN1 only?
By default, PCIe is configured for GEN2 speed. In the pcie_sample.h file, please undefine GEN2; this is to
set LINK_CAP.MAX_LINK_SPEED = 1 and PL_GEN2.DIR_SPD = 0.
4
PCI Express (PCIe) FAQ for KeyStone™ Devices
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
Copyright © 2016–2017, Texas Instruments Incorporated
How do I limit the PCIe to one lane only?
www.ti.com
16
How do I limit the PCIe to one lane only?
By default, PCIE is configured for two lane. Set PL_LINK_CTRL.LNK_MODE = 0x1 and PL_GEN2.LN_EN
= 0x1 by calling APIs.
17
How do I check the lane width and speed for the PCIe established link?
Check LINK_STAT_CTRL.LINK_SPEED: 1 for GEN1 and 2 for GEN2; LINK_STAT_CTRL.
NEGOTIATED_LINK_WD: 1 for one lane and 2 for two lane. Those fields are valid after link is up.
18
Can I run an application in DDR3 using PCIe boot?
In PCIe boot mode, the DDR3 is not initialized. Two steps are required for this purpose: first, the host
writes the DDR3 initialization code (code in L2 memory) to DSP through PCIe and lets it run to initialize
DDR3. Then, the host writes the real application into DDR3 of DSP memory through PCIe and lets it run.
A hello world example is available under pdk_c665x/7x_2_0_x\packages\ti\boot\examples\pcie.
19
Can I have multiple cores to access PCIe data space at the same time, but in
different locations?
Yes, it is allowed. The PCIe has buffers to hold those Read/Write commands.
20
What is the PCIe data space limitation?
The PCIE data space is limited to 256MB (0x6000_0000 to 0x6FFF_FFFF for Keystone I and
0x5000_0000 to 0x5FFF_FFFF for Keystone II). There are 32 outbound translation regions, each with an
8-MB maximum. If a mapping more than 256MB is required, the remote side can use different inbound
mapping setups. The same outbound read or write in the PCIe data space lands on different memory
regions on the remote side.
21
How do I test PCIe PHY loopback?
The PCIe driver example only tests the PCIe communication between two devices; there is no loopback
support. However, there may be requirements to do PCIe PHY loopback tests for diagnostic or problem
isolation purposes. See the PHY Loopback section of KeyStone Architecture Peripheral Component
Interconnect Express (PCIe) for more information.
Before starting link training, the Serdes must be in loopback for KeyStone I devices:
1. Set SERDES_CFG0.TX_LOOPBACK = 0x2, SERDES_CFG0.RX_LOOPBACK = 0x3, and
SERDES_CFG0.RX_LOS = 0. Do the same if Serdes lane 1 is also used.
After starting the link training, but before checking the link training status:
2. Skip the detect state in LTSSM and force the link to begin with the POLL_ACTIVE state by setting:
PL_FORCE_LINK.LINK_STATE = 0x2 and PL_FORCE_LINK.FORCE_LINK = 0x1.
3. The DEBUG0.LTSSM_STATE is expected to be 0x11 for link up. To pass the data exchange test, the
value of PCIE_OB_LO_ADDR_RC or PCIE_IB_LO_ADDR_RC must be changed to be equal to each
other in the pcie_sample.h file.
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
PCI Express (PCIe) FAQ for KeyStone™ Devices
Copyright © 2016–2017, Texas Instruments Incorporated
5
Resources
22
www.ti.com
Resources
The following list provides documentation and hardware resources, and lists the associated platform or
device.
1. KeyStone Architecture Peripheral Component Interconnect Express (PCIe), (SPRUGS6)
2. PCIe Use Cases for KeyStone Devices, (SPRABK8)
3. Throughput Performance Guide for KeyStone II Devices, (SPRABK5)
4. EVM6678 EVM Page, C6678
5. EVM6670 EVM Page, C6670
6. EVM6657 EVM Page, C6657
7. EVM66AK2Hx EVM Page, 66AK2Hx
8. EVM66AK2Ex EVM Page, 66AK2Ex
9. TMDXEVM6678L EVM Hardware Setup, C6678
10. TMDXEVM6670L EVM Hardware Setup, C6670
11. TMDSEVM6657L EVM Hardware Setup, C6657
12. EVMK2H Hardware Setup, 66AK2Hx
13. EVMK2E Hardware Setup, 66AK2Ex
The following list provides PCIe software resources and shows the supported devices.
1. PROCESSOR-SDK-RTOS-C665x 03_00_00_04, C6657
2. PROCESSOR-SDK-RTOS-C667x 03_00_00_04, C6670, C6678
3. Processor SDK RTOS Getting Started Guide, C6657, C6670, C6678, 66AK2Hx, 66AK2Ex, AM5K2Ex
4. PROCESSOR-SDK-RTOS-K2HK 03_00_00_04, 66AK2Hx
5. PROCESSOR-SDK-RTOS-K2E 03_00_00_04, 66AK2Ex, AM5K2Ex
6. DESKTOP-LINUX-SDK 01_00_03_00, C6678
7. Desktop-linux-sdk 01.00.00.04 Getting Started Guide, C6678
6
PCI Express (PCIe) FAQ for KeyStone™ Devices
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
Copyright © 2016–2017, Texas Instruments Incorporated
Revision History
www.ti.com
Revision History
NOTE: Page numbers for previous revisions may differ from page numbers in the current version.
Changes from Original (November 2016) to A Revision ................................................................................................ Page
•
•
•
•
•
•
•
•
Added How do I disable active state power management (ASPM)? section ......................................................
Added How do I limit the PCIe speed to GEN1 only? section ......................................................................
Added How do I limit the PCIe to one lane only? section............................................................................
Added How do I check the lane width and speed for the PCIe established link? section .......................................
Added Can I run an application in DDR3 using PCIe boot? section ...............................................................
Added Can I have multiple cores to access PCIe data space at the same time, but in different locations? section ........
Added What is the PCIe data space limitation? section .............................................................................
Added How do I test PCIe PHY loopback? section ...................................................................................
SPRAC59A – November 2016 – Revised May 2017
Submit Documentation Feedback
Copyright © 2016–2017, Texas Instruments Incorporated
Revision History
4
4
5
5
5
5
5
5
7
IMPORTANT NOTICE FOR TI DESIGN INFORMATION AND RESOURCES
Texas Instruments Incorporated (‘TI”) technical, application or other design advice, services or information, including, but not limited to,
reference designs and materials relating to evaluation modules, (collectively, “TI Resources”) are intended to assist designers who are
developing applications that incorporate TI products; by downloading, accessing or using any particular TI Resource in any way, you
(individually or, if you are acting on behalf of a company, your company) agree to use it solely for this purpose and subject to the terms of
this Notice.
TI’s provision of TI Resources does not expand or otherwise alter TI’s applicable published warranties or warranty disclaimers for TI
products, and no additional obligations or liabilities arise from TI providing such TI Resources. TI reserves the right to make corrections,
enhancements, improvements and other changes to its TI Resources.
You understand and agree that you remain responsible for using your independent analysis, evaluation and judgment in designing your
applications and that you have full and exclusive responsibility to assure the safety of your applications and compliance of your applications
(and of all TI products used in or for your applications) with all applicable regulations, laws and other applicable requirements. You
represent that, with respect to your applications, you have all the necessary expertise to create and implement safeguards that (1)
anticipate dangerous consequences of failures, (2) monitor failures and their consequences, and (3) lessen the likelihood of failures that
might cause harm and take appropriate actions. You agree that prior to using or distributing any applications that include TI products, you
will thoroughly test such applications and the functionality of such TI products as used in such applications. TI has not conducted any
testing other than that specifically described in the published documentation for a particular TI Resource.
You are authorized to use, copy and modify any individual TI Resource only in connection with the development of applications that include
the TI product(s) identified in such TI Resource. NO OTHER LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE TO
ANY OTHER TI INTELLECTUAL PROPERTY RIGHT, AND NO LICENSE TO ANY TECHNOLOGY OR INTELLECTUAL PROPERTY
RIGHT OF TI OR ANY THIRD PARTY IS GRANTED HEREIN, including but not limited to any patent right, copyright, mask work right, or
other intellectual property right relating to any combination, machine, or process in which TI products or services are used. Information
regarding or referencing third-party products or services does not constitute a license to use such products or services, or a warranty or
endorsement thereof. Use of TI Resources may require a license from a third party under the patents or other intellectual property of the
third party, or a license from TI under the patents or other intellectual property of TI.
TI RESOURCES ARE PROVIDED “AS IS” AND WITH ALL FAULTS. TI DISCLAIMS ALL OTHER WARRANTIES OR
REPRESENTATIONS, EXPRESS OR IMPLIED, REGARDING TI RESOURCES OR USE THEREOF, INCLUDING BUT NOT LIMITED TO
ACCURACY OR COMPLETENESS, TITLE, ANY EPIDEMIC FAILURE WARRANTY AND ANY IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL
PROPERTY RIGHTS.
TI SHALL NOT BE LIABLE FOR AND SHALL NOT DEFEND OR INDEMNIFY YOU AGAINST ANY CLAIM, INCLUDING BUT NOT
LIMITED TO ANY INFRINGEMENT CLAIM THAT RELATES TO OR IS BASED ON ANY COMBINATION OF PRODUCTS EVEN IF
DESCRIBED IN TI RESOURCES OR OTHERWISE. IN NO EVENT SHALL TI BE LIABLE FOR ANY ACTUAL, DIRECT, SPECIAL,
COLLATERAL, INDIRECT, PUNITIVE, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES IN CONNECTION WITH OR
ARISING OUT OF TI RESOURCES OR USE THEREOF, AND REGARDLESS OF WHETHER TI HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
You agree to fully indemnify TI and its representatives against any damages, costs, losses, and/or liabilities arising out of your noncompliance with the terms and provisions of this Notice.
This Notice applies to TI Resources. Additional terms apply to the use and purchase of certain types of materials, TI products and services.
These include; without limitation, TI’s standard terms for semiconductor products http://www.ti.com/sc/docs/stdterms.htm), evaluation
modules, and samples (http://www.ti.com/sc/docs/sampterms.htm).
Mailing Address: Texas Instruments, Post Office Box 655303, Dallas, Texas 75265
Copyright © 2017, Texas Instruments Incorporated
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising