Texas Instruments | Implementing a Fast 3-D Vision Sensor With Multiple TMS320C31 DSPs | Application notes | Texas Instruments Implementing a Fast 3-D Vision Sensor With Multiple TMS320C31 DSPs Application notes

Texas Instruments Implementing a Fast 3-D Vision Sensor With Multiple TMS320C31 DSPs Application notes
Disclaimer: This document was part of the First
European DSP Education and Research Conference.
It may have been written by someone whose native
language is not English. TI assumes no liability for the
quality of writing and/or the accuracy of the
information contained herein.
Implementing a Fast 3-D Vision with
Multiple TMS320C31 DSPs
Authors: Oksenhendler V., Miche P., Bensrhair A.
ESIEE, Paris
September 1996
SPRA331
IMPORTANT NOTICE
Texas Instruments (TI) reserves the right to make changes to its products or to discontinue any
semiconductor product or service without notice, and advises its customers to obtain the latest version of
relevant information to verify, before placing orders, that the information being relied on is current.
TI warrants performance of its semiconductor products and related software to the specifications applicable
at the time of sale in accordance with TI’s standard warranty. Testing and other quality control techniques
are utilized to the extent TI deems necessary to support this warranty. Specific testing of all parameters of
each device is not necessarily performed, except those mandated by government requirements.
Certain application using semiconductor products may involve potential risks of death, personal injury, or
severe property or environmental damage (“Critical Applications”).
TI SEMICONDUCTOR PRODUCTS ARE NOT DESIGNED, INTENDED, AUTHORIZED, OR WARRANTED
TO BE SUITABLE FOR USE IN LIFE-SUPPORT APPLICATIONS, DEVICES OR SYSTEMS OR OTHER
CRITICAL APPLICATIONS.
Inclusion of TI products in such applications is understood to be fully at the risk of the customer. Use of TI
products in such applications requires the written approval of an appropriate TI officer. Questions concerning
potential risk applications should be directed to TI through a local SC sales office.
In order to minimize risks associated with the customer’s applications, adequate design and operating
safeguards should be provided by the customer to minimize inherent or procedural hazards.
TI assumes no liability for applications assistance, customer product design, software performance, or
infringement of patents or services described herein. Nor does TI warrant or represent that any license,
either express or implied, is granted under any patent right, copyright, mask work right, or other intellectual
property right of TI covering or relating to any combination, machine, or process in which such
semiconductor products or services might be or are used.
Copyright © 1997, Texas Instruments Incorporated
TRADEMARKS
TI is a trademark of Texas Instruments Incorporated.
Other brands and names are the property of their respective owners.
CONTACT INFORMATION
US TMS320 HOTLINE
(281) 274-2320
US TMS320 FAX
(281) 274-2324
US TMS320 BBS
(281) 274-2323
US TMS320 email
dsph@ti.com
Contents
Abstract ........................................................................................................................... 7
Product Support on the World Wide Web .................................................................... 8
Stereovision Process ..................................................................................................... 9
System ........................................................................................................................... 10
Advanced Image Manager........................................................................................ 10
Computational Module Controller ............................................................................. 10
Programmable Accelerator Computational Module .................................................. 11
External Communication Devices ............................................................................. 11
Tasks Allocation ........................................................................................................... 15
Results........................................................................................................................... 20
Summary ....................................................................................................................... 23
References .................................................................................................................... 23
Figures
Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.
Figure 7.
Figure 8.
CM_PA Block Diagram..................................................................................... 12
General Configuration ...................................................................................... 13
Tasks Allocation and Data Transfer ................................................................. 15
Chronogram...................................................................................................... 17
Synchronization by serial link ........................................................................... 19
Left Image......................................................................................................... 20
Right Image ...................................................................................................... 21
Depth Map ........................................................................................................ 22
Implementing a Fast 3-D Vision
Sensor with Multiple TMS320C31
DSPs
Abstract
The stereovision process determines the distances of objects by
acquiring two images in different positions and studying the
difference in locations of the corresponding stereo points. Various
techniques have been developed to infer 3-D information from a
set of brightness images. One technique, called passive
stereovision, is appealing because of its ranging applications and
especially because of its ability to work in various illumination
conditions and with a large depth range (HOR 91).
This application report describes an automatic stereovision
process with mobile robot guidance and autonomous vehicle
navigation. Because the performance of this system requires highspeed response, the stereovision algorithms are implemented on
specialized architecture that includes three Texas Instruments
(TI™) TMS320C31 digital signal processors (DSPs). The 150/40vision system from Imaging Technology has been chosen to
implement the stereovision algorithms because it is a modular
system that allows the parallelism of many image treatments.
Experimental results are presented at the end of this paper.
This document was part of the first European DSP Education and
Research Conference that took place September 26 and 27, 1996
in Paris. For information on how TI encourages students from
around the world to find innovative ways to use DSPs, see TI’s
World Wide Web site at www.ti.com.
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
7
SPRA331
Product Support on the World Wide Web
Our World Wide Web site at www.ti.com contains the most up to
date product information, revisions, and additions. Users
registering with TI&ME can build custom information pages and
receive new product updates automatically via email.
8
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Stereovision Process
In general, the stereovision process consists of three steps:
q
q
q
Feature extraction in right and left images
Feature matching
Calculation of the 3-D map
To reduce computation time and simplify the stereovision
algorithms, we chose a special configuration. The two cameras
have parallel optical axis and the ith scanline of the right and left
CCDs is in the exact extension of each other. Given this
configuration, the epipolar lines are confused with the scanlines of
the CCDs. Consequently, the stereovision algorithms can be
processed line by line.
The first step of our stereovision system is based on a new
concept that we call declivity. Three-dimensional reconstruction is
based on the detection and matching of characteristic declivities.
In an image line, they are detected using an automatic threshold
[MIC 94]. Furthermore, to be matched, this primitive is
characterized by its position and the photometric characteristic of
its neighborhood.
To match declivities, the second step of the system is based on a
new matching algorithm, which uses dynamic programming
method [BEN 96]. The algorithm parameters depend on those
calculated on the detection phase of declivities.
Once the matching declivities are found, the 3-D information is
deduced using disparity values and the depth map is completed
by interpolation.
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
9
SPRA331
System
The system uses the 150/40 VME vision system from Imaging
Technology. This modular system allows general-purpose image
treatment and specific processes depending on the chosen
configuration. It is composed of motherboards at the VME format
on which can be plugged specific modules.
Our configuration is composed of two motherboards: an Advanced
Image Manager (IMA) and a Computational Module Controller
(CMC), one of which is a plugged module. (See Figure 2.) The
motherboards include the following modules:
q
q
q
Color Acquisition Module (AM_CLR) plugged into the IMA
Pseudo Color Display Module (DM-PC) plugged into the IMA
Three Programmable Accelerator Computational Modules
(CM_PA) based on the TI floating point TMS320C31 DSP; one
plus into the IMA and the others plug into the CMC
Advanced Image Manager
The IMA contains four Megabytes of reconfigurable image
memory, a cross-port switch for data routing, and three supports
for plug-in modules.
Memory is organized into four frames of 1K x 1K x 8 bits. Each
frame is provided with two asynchronous 8-bit video ports: one for
input and the other for output. In our case, we use only three
frames: one for display and two for acquisition from cameras.
The cross-port switch is a specific circuit that allows configuring
connections between the six 8 bit inputs and the six 8 bit outputs
of IMA, and inputs and outputs of IMA modules.
Computational Module Controller
As for the IMA, it provides supports for three plug-in computational
modules and a cross-port switch to configure data communication
for modules.
The cross-port switch allows configuring video connections with
external boards: six 8 bit inputs and three 8-bit outputs as well as
video connections between modules of the CMC.
The CMC also provides local connections between serial ports of
computational modules.
10
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Programmable Accelerator Computational Module
The CM_PA is based on the floating point TMS320C31 DSP (see
Figure 1.) The TMS320C31 has an addressing space of 24 bits, a
32 bits data width, two 2K x 32 bits single cycle RAM, a 64 x 32
bits instruction cache, a single cycle floating point multiplier, a
single cycle ALU, two address generators, and a DMA controller.
The DSP addresses four different types of external memory:
q
Program memory
128 x 32 bits of zero wait state static RAM for program store,
extended stack, coefficient storage, and general purpose
memory.
q
Dual port RAM
4K x 16-bit memory shared by the DSP and the CPU host
allows communications without halting the DSP. In our
process, it is used only during initialization when the host loads
program code and configures the CM_PA.
q
Image memory:
1M x 32 bits addressable by the DSP, by the VME host, and
by the video data bus. Image memory cannot be accessed
simultaneously by a different process. Access time is 1 wait
state for access in the same row of 1K wide and 3 wait states
in different rows.
q
Flash EEPROM boot memory
256K x 32 bits that allows the TMS320C31 to boot itself. If a
system is composed only of IMAs with one CM_PA each, flash
EEPROM memory allows it to be independent of the host. This
memory is not used in our 3-D vision sensor application.
External Communication Devices
The following external communication devices are included:
q
q
Dual port RAM
One 16-bit video input port used to connect to the video bus.
This port is not separable in two 8-bit ports but the use of only
8 bits is possible. The same characteristics are available for
video output.
q
One serial link
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
11
SPRA331
Figure 1. CM_PA Block Diagram
Figure 2 shows the general scheme of the 3-D vision sensor
system.
12
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Figure 2. General Configuration
The host at the initialization configures all of the system. In our
case, the CM_PA of the IMA is the server unit. During the
computation process, it controls the cross-port switch and the
modules of the IMA.
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
13
SPRA331
The Imaging Technology vision system includes software utilities
and libraries used to configure boards and modules and manage
computation of CM_PAs. The C program language is used both
for the host and DSP. Debugging the DSP program is possible
only for the CM_PA of IMA when it is not a server. To develop the
3-D sensor programs, the serial port of CM_PAs controls data
flows and computations.
14
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Tasks Allocation
To take advantage of the three DSPs, the stereovision process is
separated into tasks distributed as follows (see Figure 3):
q
q
The two DSPs on the CMC are allocated for right and left
image segmentation.
The DSP on the IMA (which is the server) matches
characteristic elements issued from DSPs of the CMC and
manages acquisition and data transfer.
Figure 3. Tasks Allocation and Data Transfer
Data communications are made via the video buses. All data
transferred by video buses must have the same image format,
here imposed by the acquisition, which are 512 x 512 x 8 bits. The
data transfer step is time critical due to the amount of data
communications.
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
15
SPRA331
During a loop (excluding the first one since it is particular), right
and left images are acquired synchronously in the IMA frame
memory during the matching of characteristic elements issued of
the previous stereo image pair. (See Figure 4.) This implies no
time penalty because the acquisition in IMA memory frame is
managed by the IMA board without halting the DSP.
Next, images are transferred from the IMA memory frame to the
image memories of CM_PAs of the CMC. The DSPs must wait for
the end of image loading before extracting declivities.
Characteristic elements are then converted into an image format,
loaded in image memory, and dumped to the video bus connected
to the IMA board.
To get back characteristic elements, the DSP of the IMA connects
its entry port to the video bus linked with one of the DSPs of the
CMC and acquires the characteristic element image. Then it
connects its entry port to the video bus linked with the other DSP
of the CMC and acquires the other characteristic elements.
Characteristic element images are converted and loaded in DSP
RAM (which is a one wait state memory) and matched to calculate
the 3-D map. Results are then converted into an image format,
written in DSP image memory, and dumped to the display.
This process is looped in each DSP as shown in Figure 4 and
Figure 5.
16
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Figure 4. Chronogram
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
17
SPRA331
The serial link is dedicated to the synchronization task. Before the
beginning of the stereo process, the DSPs must wait for the end of
the load of the program code in each DSP. For this, the IMA DSP
emits a zero on the serial link to the first CMC DSP. Next, it emits
the received number incremented by one until a three is emitted
and received. In case of erroneous transmission (reception of a
number different of 0, 1, 2, or 3) the process comes back to the
beginning by emitting a zero.
For synchronization, the same process is used for all of the
various tasks, acquisition, segmentation, and matching.
Synchronization words corresponding at the beginning or at the
end of each task are transmitted by the serial link.
18
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Figure 5. Synchronization by serial link
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
19
SPRA331
Results
The sensor was tested with indoor and outdoor scenes. The rate
of the sensor is calculated from 200 successive processes on
images, which have a format of 512 x 512 x 8 bits. The
computation times comprise between 1 and 1.5 seconds,
depending on the scene complexity. The following result
corresponds to an indoor scene involving a chair in front of a white
board. (See Figure 6, Figure 7, and Figure 8.)
Figure 6. Left Image
20
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Figure 7. Right Image
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
21
SPRA331
Figure 8. Depth Map
In the resulting depth map, dark pixels mean far points and white
pixels mean near points.
The computation time for this scene is about 1.1 seconds. This
allows time to consider specific applications, such as robot
guidance.
22
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
SPRA331
Summary
Results obtained with the 150/40-vision system from Imaging
Technology show that the Texas Instruments TMS320C31 DSPs
are well suited for our 3-D vision sensor algorithms. We hope that
new DSP generations, such as the Texas Instruments
TMS320C80 will significantly reduce processing time. A system
with fast shared memory will exchange data (characteristic
elements) without needing any image format conversion.
Moreover, the treatments can be made line by line allowing a high
parallelism level.
References
[BEN 96]: Bensrhair A., Miché P. and Debrie R., (1996). “Fast and
automatic stereovision matching algorithm based on
dynamic programming method.” Pattern Recognition
Letters, 17, 457-466.
[HOR 91]: Horn B. K. P., (1991). Robot Vision. MIT Press, 299-326.
[MIC 94]: Miché P. and Debrie R., (1994). “Fast and Self-Adaptive Image
Segmentation Using Extended Declivity.” Annals of
Telecommunications, 50, n03-4, 401-410.
Implementing a Fast 3-D Vision with Multiple TMS320C31 DSPs
23
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising