Keysight Technologies Tips to Achieve PCIe Connectivity with your

Keysight Technologies Tips to Achieve PCIe Connectivity with your
Keysight Technologies
Tips to Achieve PCIe® Connectivity
with your Keysight AXIe or PXIe Chassis
Application Brief
Achieve PCIe Connectivity
The Keysight Technologies, Inc. AXIe and PXIe
chassis use PCIe as their primary communications link. Using the PCIe bus has many advantages such as availability of standard hardware,
signaling methods and debug tools.
The focus of this technical note is to provide
a guide to address PCI/PCIe communications
issues if they occur while using the Keysight
AXIe or PXIe chassis products with external PC
controllers.
OVERVIEW
Introduction
Contents
Tip 1: Verify connections are fully seated
4
Tip 2: Understand and adhere to proper
power sequencing
4
Tip 3: Verify PCIe bus connectivity via LED
indicators
5
Tip 4: Understand and correct WDM error
codes
7
Keysight Technology, Inc’s AXIe and PXIe chassis use
PCIe as their primary communications link. Using the
PCIe bus has many advantages such as availability of
standard hardware, signaling methods and debug tools.
Despite these advantages you can run into cases where
chassis and/or instruments have issues establishing
communicating to the PC controller. The focus of this
technical note is to provide a guide to address PCI/PCIe
communications issues if they occur while using the
Keysight AXIe or PXIe chassis products with external PC
controllers. PCI/PCIe connectivity is best indicated by
viewing the Microsoft Windows Device Manager (WDM)
as shown in Figure 1.
Tip 5: Review and reset PCIe fabrics if
necessary (PXIe M9018A only)
7
When to use this guide
Tip 6: Understand and verify available PCI
bus endpoints
8
Tip 7: Upgrade the PCIe adapter (non-laptop
controller applications)
9
Tip 8: Select different controller PC slot for
plug-in adaptor
10
Tip 9: Selectively remove modules
10
Overview
2
Introduction
2
Understanding debug tools and general troubleshooting flow
4
Overview of debug process
4
Summary
This guide will be helpful if the chassis and some or all of
the modules are not present and viewable in WDM (Figure
1a). In addition, this guide will discuss situations where a
yellow icon appears next to the PXI module(s) as it appears
in WDM (Figure 1b).
When not to use this guide
10
This guide is not usually helpful if the all the modules are
viewable in WDM (Figure 1c). The correct appearance in
WDM indicates the instruments are present on the PCI
bus and the instrument drivers are installed. If this is the
case, verify the Keysight I/O libraries are installed correctly
www.keysight.com/find/iosuite and then verify instrument
communications using the soft front panel (provided with
the instrument driver). Consult the instrument specific
documentation for further debug assistance if needed.
2
OVERVIEW
Figure 1a. Windows Device Manager does
not show chassis or modules even after
verifying chassis is properly cabled to
controller.
Figure 1b. Windows Device Manager shows Figure 1c. Windows Device Manager
shows all modules and device (instruonly some modules and/or the modules
ment) drivers are installed correctly.
appear with a yellow icon.
This guide is intended to supplement the AXIe and
PXIe chassis start-up and users guides. Portions of this
document may refer to procedures described in the
documents:
First and foremost it is recommended using a controller
that is pre-tested by Keysight Technologies:
– See document 5990-7632EN Tested Computer List
Technical Note available at www.keysight.com.
– M9018A startup and user guides:
The pre-tested controllers have been determined to have
enough PCI bus resources to enumerate at least one full
sized chassis. In cases where a pre-tested controller is
not available, this guide will give you information that will
help determine if your controller will work in your particular
configuration.
- P/N M9018-90001
- P/N M9018-90005
– M9502A/05A startup and user guides:
- P/N M9502-90001
- P/N M9502-90005
3
UNDERSTANDING DEBUG TOOLS AND GENERAL TROUBLESHOOTING
– Be sure the PCIe cable connectors—and ExpressCard
if used—are fully engaged. A click will be heard when
they are seated properly.
– Verify all AXIe or PXIe modules are fully seated. The
engagement of the ejection lever is not always a good
indication of full engagement. A better test is to verify
the module faceplate screws can be tightened with
little or no binding of the faceplate sheet-metal. Also,
be aware that some AXIe or PXIe modules may have
vinyl screw caps on the faceplate screws. Make sure
these are removed before installing the module.
The PCI/PCIe bus is a prevalent interface standard used
world-wide. As such many tools and methods are available that can be used to inspect the status of the bus. For
example, Microsoft Windows Device Manager (WDM) is a
good tool to use when initially inspecting and understanding the current state of the PCI bus. Other tools by 3rd
party software providers also can give independent reports
on PCI resources which are helpful and will be referenced
in this document.
Issues with PCI/PCIe connectivity usually fall into a few
categories: incorrect or missing driver, limited available
PCI resources or signal equalization and clock jitter when
driving cables. Below is a summary of debugging process
for connectivity issues. Slight variations will occur
depending if you are working with AXIe or PXIe, although
basic flow is similar.
Tip 2: Understand and adhere to
proper power sequencing
Overview of debug process
The PCI endpoints (including the chassis and modules)
must be present on the PCI bus when the PC controller
boots in order for them to be identified. As a result the
proper controller/chassis start-up sequence must be used.
The proper power sequence is: (1) power up the chassis
and then; (2) power up the PC controller.
– Tip 1: Verify the connectors and modules
are fully seated.
– Tip 2: Understand and adhere to proper
chassis/controller power sequencing.
– Tip 3: Verify PCIe bus connectivity via LED
indicators.
– Tip 4: Understand and correct WDM error codes.
– Tip 5: Review and reset PCIe fabrics if necessary
(PXIe M9018A only).
– Tip 6: Understand and verify suficient number
of free PCI bus numbers.
– Tip 7: Upgrade the PCIe adaptor card if necessary.
– Tip 8: Select a different controller PCIe slot for
plug-in adaptor.
– Tip 9: Selectively remove modules.
Care must be taken to provide enough time for chassis
to be ready. The M9018A PXIe chassis powers up quickly
(less than 1 second). However the AXIe chassis must
be powered up for several seconds prior to powering the
external controller. For AXIe chassis use the ESM status
light as an indicator--it must be green before powering up
the external controller (Figure 2). Also be aware that some
AXIe modules may require additional time to fully initialize
and be present on the PCI bus.
If the power up sequence is not correct the chassis will
not be recognized by PC. Power-down both the chassis
and controller and then try again using correct sequence.
Tip 1: Verify the connectors and
modules are fully seated
The PCIe cable connectors and module mechanics are
robust and well designed, but it is surprisingly common for
them to not be fully engaged. In addition, if using a Laptop
controller the ExpressCard can easily become disengaged
from the controller with a minor bump. Pay particular
attention to the following:
Figure 2. AXIe status light on ESM module must be green before
powering-up controller.
4
OVERVIEW OF DEBUG PROCESS
Tip 3: Verify PCIe bus connectivity
via LED indicators
LED’s located on the M9021A cable card and the M9505A
ESM module provides indication if the PCIe bus is connected properly, as well as other interface card status
information.
PXIe chassis interconnect
On the M9021A card there are three groups LED’s to pay
attention to. The M9021A front panel has 4 LED’s as
shown in Figure 3a. These give an easily viewable high
level indication of PCIe lane connection status depending
on your particular configuration. If needed, additional
information can be attained by looking at the LED’s located
on the top surface of the M9021A cable card PCA. The
Blue LED’s DS301 and DS501 indicate power is applied to
the M9021A card. In addition, there are two groups of 8
LED’s giving greater insight to the individual lane connections to the backplane and cable.
Figure 3a. Front top of M9021A card PCIe LED (green) indicators.
Figure 3b. Side view of M9021A card showing additional PCIe lane
indicators (yellow) and power LED’s (blue).
5
OVERVIEW OF DEBUG PROCESS
Tip 3: Verify PCIe bus connectivity
via LED indicators (continued)
Verifying the M9021A switches and LED’s
– First, verify the M9021A switches S301 & S201 are set
correctly (located on the top side of the M9021A PCA).
They must be set to the right for proper operation in
slot 1 of the M9018A.
– Next, plug the M9021A card into the chassis, connect
the PCIe cables and power up the system. Check the
front panel LED’s irst. The LED’s will be illuminated
depending on your coniguration:
- PCIe x1 and x4 lane conigurations will
illuminate the x4 Cable and Chassis LED’s
- PCIe x8 conigurations will illuminate both
the x4 and x8 Cable and Chassis LED’s.
If the LED’s are not lit, re-check the cabling and
- inspect the PCA located LED’s as described below.
Figure 4. Backplane switch—M9021A power slide switch—must
be set correctly (M9018A only).
– Inspect the blue LED’s DS301 and DS501. They must
be lit indicating power is applied to M9021A. If the
blue LED’s are not lit, set power switch as shown in
Figure 4.
AXIe chassis interconnect
Check that the PCIe Indicator LED’s are illuminated on the
ESM PCA (AXIe M9505A system module Figure 5). These
LEDs indicate the connected PCIe lanes. IF the LED’s are
not illuminated check PCIe cable connectors to verify they
are fully seated. Also, check the adaptor card located in
the PC controller to verify it is seated.
Figure 5. AXIe chassis: LED’s on M9505A ESM indicate PCIe bus
connectivity.
6
OVERVIEW OF DEBUG PROCESS
Tip 4: Understanding and correcting WDM error codes
A yellow icon next to the PXI device as shown in WDM
(Figure 1b) may result from either a missing or corrupt
instrument driver or limited PCI resources. Inspecting
the error code will give additional information on the root
cause. Double click on the device showing the error and
view the Device status window (Figure 6). There are many
device status codes, but the most common codes associated with chassis communications are 12, 28 and 31.
Codes 28 and 31:
– Code 28: The drivers for this device are not installed.
– Code 31: This device is not working properly because
Windows cannot load the drivers required for this
device.
These codes occur when the instrument drivers are not
installed or when windows cannot find the device information file (.inf). In this case verify the instrument driver has
been correctly installed. It may be necessary to uninstall
and then re-install the instrument driver followed by a PC
re-boot. If you are uncertain if you have the correct instrument driver see the Keysight website www.keysight.com/
find/drivers and search for the appropriate IVI driver to be
sure.
Figure 6. Windows Device Manager error code.
Tip 5: Review and reset PCIe fabrics
if necessary (PXIe M9018A only)
The M9018A chassis has adjustable PCIe fabric configurations that can be modified and optimized when using an
Keysight embedded controller. However, when using an
external PC controller only the factory default PCIe 1x8
fabric is compatible with the M9021A interface cable card.
Code 12:
– Code 12: This device cannot ind enough free
resources to use.
If a previous user had configured the M9018A to a mode
other than the 1x8 factory default, you should take steps
to configure the chassis back to the default. This is most
easily done using the “PCIe Switch Fabric Configurator”
tool run on the previous (embedded) controller used with
the chassis. See the User anual Chapter 14 Changing and
Restoring the PCIe link Configuration for details. If necessary, a manual fabric reset operation is also documented in
the User manual.
When using large AXIe or PXIe instrument chassis or configurations that include an extension chassis--such RAID
memory--there may not be enough PCI resources available
for full chassis enumeration. In most cases the limitation
is the number of PCI bus numbers. See Tip 6: Understand
and verify available PCI bus numbers.
7
OVERVIEW OF DEBUG PROCESS
Tip 6: Understand and verify
available PCI bus numbers
Table 1. Required number of buses (single chassis).
Standard root complex implementations of PCI Express
allow for 256 bus numbers. However, some consumer
and business PC vendors limit the number of available bus
numbers in the system BIOS to minimize implementation
complexity. Full sized AXIe and PXIe chassis (especially
the 18-slot M9018A chassis) can consume bus numbers
quite rapidly. Keep in mind that each PXIe/AXIe peripheral
module slot in the modular chassis has a reserved bus
number and several bus numbers to support the PCIe
switches on the backplane. Depending on which particular
chassis is used, up to 30 bus numbers may be required
per chassis (does not include the bus numbers for PCIe
devices within the controller itself).
Chassis P/N
Minimum required Free
PCI bus #’s
M9018A
30
M9502A
15
M9505A
15
Keep in mind if you are attempting to use multiple chassis
you need to increase the number of buses accordingly. In
addition, if you use other components such as a PCIe RAID
expansion chassis the required number of resources will
increase as well.
A software tool provided by an independent 3rd party is
useful to check the suitability of the BIOS to support enumerating modular chassis. The tool is known as System
Information Viewer (SIV) and is provided by Ray Hinchliffe
(see http://rh-software.com). Here are the steps for BIOS
capability inspection using the SIV tool (use SIV tool version 4.28 and later):
If the available buses are a limiting factor, check with your
controller supplier to see if there are any BIOS updates.
In some cases PC suppliers have increased the number
of available PCI bus numbers and provided updates via
updated BIOS installation. It may be helpful to check the
Keysight FTP site ftp://ftp.keysight.com/mpusup/BIOS,
for BIOS updates.
On the network, go to the site: http://rh-software.com
If the number of free endpoints is a limiting factor and
there are no BIOS updates, it will be necessary to use a
different controller with your chassis. See document number 5990-7632EN: Tested Computer List Technical Note
available at www.keysight.com.
– For Windows XP and later, download the siv.zip ile
and then unzip into a directory of your choice.
– Run SIV32X.exe or SIV64X.exe depending on your OS.
– In the main window that appears, right click on the
“PCI Bus” and select ACPI busses.
– You will see a window that appears as shown below.
The key attribute to look for is the Free column. These
are the free PCI bus numbers available for enumeration.
For example, in Figure 7, we see 216 free buses are
available.
– Compare the number of free bus endpoints available
to table 1. If the number of busses is greater than
listed, the BIOS should be capable of enumerating the
chassis. It is always best to have a large number of
extra buses to allow for expansion.
If the available buses are not a limiting factor, the next
likely cause is related to the signal characteristics of the
PCIe Express bus over a cable. The next few steps will
help identify issues with PCIe signals over the cables.
Figure 7. Screenshot showing SIV window.
8
OVERVIEW OF DEBUG PROCESS
Tip 7: PCIe adapter considerations
The PCIe bus runs at very high data rates and when controlling an external chassis the PCIe bus must be routed
from the PC controller to the chassis using cables. Access
to the controller’s internal PCIe bus is achieved using
adaptor hardware which takes the form of either a PCIe
plug-in adaptor card (Figure 8) or an ExpressCard adaptor
(Figure 9).
Some PCIe adaptors contain re-driver hardware that is
only capable to PCIe Gen 1 data rates. This presents a
problem when a PCIe Gen 2 capable PC is connecting to a
Gen 2 capable chassis since the PCIe bus will attempt to
negotiate at Gen 2 rates (both endpoints will advertise Gen
2 capability). If the PCIe cable adaptor is only designed
for Gen 1 rates it may not be able to pass the Gen 2 data
rates, as needed, and link training will fail.
For best performance use a PCIe adaptor that is engineered to support the required data rates including optimized signal drive equalization and clock isolation. Here
are some attributes to look for depending on the adaptor
style:
Figure 8. M9048A PCIe adaptor card
PCIe adaptor plug-in card
The Keysight M9048A adaptor card has been engineered to
provide superior transmission performance at Gen 2 rates.
It is tuned to drive PCIe cables and provides clock isolation
which improves jitter performance across the cable. In the
event you have an older PCIe adaptor card, upgrade to the
Keysight M9048A adaptor. See www.keysight.com/find/
m9048a for details.
PCIe ExpressCard
It is important to note that many ExpressCards are rated
for Gen 1 rates only while some newer laptops may
support PCIe Gen 2 rates. The latter case may present a
problem if the ExpressCard itself only supports Gen 1 rates.
In cases where the Laptop PC supports PCIe Gen 2,
the BIOS will generally have an option that forces the
ExpressCard slot to run at a Gen 1 data rate. If your laptop
supports PCIe Gen 2 data rates, check the BIOS settings
and select the option to for the ExpressCard slot to negotiate a Gen 1 data rate.
Figure 9. M9045B ExpressCard
9
OVERVIEW OF
DEBUG PROCESS
SUMMARY
Tip 8: Select different controller PC
slot for plug-in adaptor
There may be cases where PXI and AXIe chassis or instruments are having issues establishing communicating with
an external to the controller. The focus of this technical
note is to provide a guide to address basic PCIe communications issues if they occur while using the Keysight PXIe
or AXI chassis products with external controllers.
PCI bus architectures vary depending on the controller.
The PCI and PCIe bus can be partitioned into multiple
sections intended for different application access. Perhaps
you have heard the term Northbridge and Southbridge
ports on the PCIe bus? Historically highspeed memory and
graphics PCI access was via the northbridge, while lower
speed peripheral access (USB, IDE…) was via the southbridge. Depending on the controller, differences may occur
between these ports that can impact external PCIe chassis
communications. For example, some older controllers may
partition available PCI bus numbers differently depending
on the port used. For this reason, it may be helpful to relocate the PCIe adaptor card (M9048A) to a different PCIe
slot location.
In some cases the solution can be as simple as re-seating
a connector. In other cases getting the appropriate PCIe
adaptor card will resolve issues. There may be some cases
where the computer BIOS is not capable of enumerating
a full chassis. In those cases it may be required to use a
different PC controller.
For additional information see:
www.keysight.com/find/M9018A
www.keysight.com/find/AXIe-chassis
Tip 9: Selectively remove modules
There may be some cases where a defective module may
prevent other modules from booting. If the above tips fail
to isolate the problem, remove all the modules in the chassis and attempt to reboot and verify connectivity to the
chassis. In the PXIe M9018A chassis, the chassis itself
appears as a PCI node in the WDM. For an AXIe M502A or
M9505A chassis, after removing all of the modules in the
chassis, the only PCIe based node that will appear is the
LAN NIC controller. This is best observed using the view
by connection mode in WDM.
If the chassis PCI node (M9018A) or the LAN NIC node
(M9502A or M9505A) appear in WDM after removing all
of the modules, selectively re-install the modules to determine the offending one.
10
12 | Keysight | Tips to Achieve PCIe Connectivity with your Agilent AXIe or PXIe Chassis - Application Brief
myKeysight
www.keysight.com/find/mykeysight
A personalized view into the information most relevant to you.
www.axiestandard.org
AdvancedTCA® Extensions for Instrumentation and Test (AXIe) is an
open standard that extends the AdvancedTCA for general purpose and
semiconductor test. Keysight is a founding member of the AXIe consortium.
www.lxistandard.org
LAN eXtensions for Instruments puts the power of Ethernet and the
Web inside your test systems. Keysight is a founding member of the LXI
consortium.
www.pxisa.org
PCI eXtensions for Instrumentation (PXI) modular instrumentation delivers a
rugged, PC-based high-performance measurement and automation system.
Three-Year Warranty
www.keysight.com/find/ThreeYearWarranty
Keysight’s commitment to superior product quality and lower total cost
of ownership. The only test and measurement company with three-year
warranty standard on all instruments, worldwide.
Keysight Assurance Plans
www.keysight.com/find/AssurancePlans
Up to five years of protection and no budgetary surprises to ensure your
instruments are operating to specification so you can rely on accurate
measurements.
www.keysight.com/quality
Keysight Electronic Measurement Group
DEKRA Certified ISO 9001:2008
Quality Management System
Keysight Channel Partners
www.keysight.com/find/channelpartners
Get the best of both worlds: Keysight’s measurement expertise and product
breadth, combined with channel partner convenience.
PICMG and the PICMG logo, CompactPCI and the CompactPCI logo,
AdvancedTCA and the AdvancedTCA logo are US registered trademarks
of the PCI Industrial Computers Manufacturers Group.“PCIe” and “PCI
EXPRESS” are registered trademarks and/or service marks of PCI-SIG.
www.keysight.com
www.keysight.com/find/pxi
www.keysight.com/find/modular
For more information on Keysight
Technologies’ products, applications or
services, please contact your local Keysight
office. The complete list is available at:
www.keysight.com/find/contactus
Americas
Canada
Brazil
Mexico
United States
(877) 894 4414
55 11 3351 7010
001 800 254 2440
(800) 829 4444
Asia Paciic
Australia
China
Hong Kong
India
Japan
Korea
Malaysia
Singapore
Taiwan
Other AP Countries
1 800 629 485
800 810 0189
800 938 693
1 800 112 929
0120 (421) 345
080 769 0800
1 800 888 848
1 800 375 8100
0800 047 866
(65) 6375 8100
Europe & Middle East
Austria
Belgium
Finland
France
Germany
Ireland
Israel
Italy
Luxembourg
Netherlands
Russia
Spain
Sweden
Switzerland
United Kingdom
0800 001122
0800 58580
0800 523252
0805 980333
0800 6270999
1800 832700
1 809 343051
800 599100
+32 800 58580
0800 0233200
8800 5009286
0800 000154
0200 882255
0800 805353
Opt. 1 (DE)
Opt. 2 (FR)
Opt. 3 (IT)
0800 0260637
For other unlisted countries:
www.keysight.com/find/contactus
(BP-05-23-14)
This information is subject to change without notice.
© Keysight Technologies, 2013-2014
Published in USA, August 2, 2014
5991-0385EN
www.keysight.com
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising