camera positioning and calibration techniques for integrating traffic

camera positioning and calibration techniques for integrating traffic
FINAL
CONTRACT REPORT
CAMERA POSITIONING
AND CALIBRATION TECHNIQUES
FOR INTEGRATING TRAFFIC SURVEILLANCE
VIDEO SYSTEMS WITH MACHINE-VISION
VEHICLE DETECTION DEVICES
BRIAN L. SMITH, Ph.D.
Assistant Professor
Department of Civil Engineering
University of Virginia
MICHAEL L. PACK
Graduate Research Asssistant
V·I·R·G·I·N·I·A
TRANSPORTATION RESEARCH COUNCIL
VIRGINIA TRANSPORTATION RESEARCH COUNCIL
1. Report No.
FHWA/VTRC 03-CR9
Standard Title Pa2e - Report on Federally Funded Project
2. Government Accession No.
3. Recipient's Catalog No.
4. Title and Subtitle
Camera Positioning and Calibration Techniques for Integrating Traffic
Surveillance Video Systems with Machine-Vision Vehicle Detection Devices
5. Report Date
December 2002
6. Performing Organization Code
7. Author(s)
Brian L. Smith and Michael L. Pack
8. Performing Organization Report No.
VTRC 03-CR9
9. Performing Organization and Address
1O. Work Unit No. (TRAIS)
Virginia Transportation Research Council
530 Edgemont Road
Charlottesville, VA 22903
11. Contract or Grant No.
60226
12. Sponsoring Agencies' Name and Address
Virginia Department of Transportation
1401 E. Broad Street
Richmond, VA 23219
15. Supplementary Notes
FHWA
P.O. Box 10249
Richmond, VA 23240
13. Type of Report and Period Covered
Final
September 200 I-October 2002
14. Sponsoring Agency Code
16. Abstract
The Virginia Department of Transportation has invested significantly in extensive closed circuit television (CCTV) systems to
monitor freeways in urban areas. Although these systems have proven very effective in supporting incident management, they do not
support the collection of quantitative measures of traffic conditions. Rather, they simply provide a moveable platform for trained
operators to collect images for further interpretation. Although there are several video image vehicle detection systems (VIVDS) on
the market that have the capability to derive traffic measures from video imagery automatically, these systems currently require the
installation of fixed-position cameras. Thus, they have not been integrated with the existing moveable CCTV cameras.
This research effort addressed VIVDS camera repositioning and calibration challenges and developed a prototype machinevision system that successfully integrates existing moveable CCTV cameras with VIVDS. Results of testing the prototype in a
laboratory setting demonstrated that when the camera's original zoom level was at a level of 1x to 1.5x, the system could return the
camera to its original position with a repositioning accuracy of less than 0.03 to 0.1 degree. This is significantly less that the 0.5degree accuracy of mechanical camera presets and indicates that such an approach provides the accuracy needed for CCTV/VIVDS
integration. This level of positional accuracy, when combined with a VIVDS, resulted in vehicle count errors of less than 1%. Based
on these results, the integration of CCTV and VIVDS is feasible, thus paving the way for less costly, more easily maintained traffic
monitoring systems in future intelligent transportation system initiatives.
17 Key Words
18. Distribution Statement
Closed circuit television (CCTV)
No restrictions. This document is available to the public through
Video image vehicle detection systems (VIVDS)
NTIS, Springfield, VA 22161.
Intelligent transportation systems (ITS)
Traffic monitoring and measurement
19. Security Classif. (of this report)
20. Security Classif. (of this page)
21. No. of Pages
22. Price
Unclassified
Unclassified
20
Form DOT F 1700.7 (8-72)
Reproduction of completed page authorized
FINAL CONTRACT REPORT
CAMERA POSITIONING AND CALIBRATION TECHNIQUES FOR INTEGRATING
TRAFFIC SURVEILLANCE VIDEO SYSTEMS WITH MACHINE-VISION VEHICLE
DETECTION DEVICES
Brian L. Smith, Ph.D.
Assistant Professor
Department of Civil Engineering
University of Virginia
Michael L. Pack
Graduate Research Assistant
Project Managers
Catherine C. McGhee, Virginia Transportation Research Council
Michael A. Perfater, Virginia Transportation Research Council
Contract Research Sponsored by
the Virginia Transportation Research Council
Virginia Transportation Research Council
(A Cooperative Organization Sponsored Jointly by the
Virginia Department of Transportation and
the University of Virginia)
Charlottesville, Virginia
December 2002
VTRC 03-CR9
NOTICE
The project that is the subject of this report was done under contract for the Virginia
Department of Transportation, Virginia Transportation Research Council. The contents
of this report reflect the views of the authors, who are responsible for the facts and the
accuracy of the data presented herein. The contents do not necessarily reflect the
official views or policies of the Virginia Department of Transportation, the
Commonwealth Transportation Board, or the Federal Highway Administration. This
report does not constitute a standard, specification, or regulation.
Each contract report is peer reviewed and accepted for publication by Research Council
staff with expertise in related technical areas. Final editing and proofreading of the
report are performed by the contractor.
Copyright 2002 by the Commonwealth of Virginia.
11
ABSTRACT
The Virginia Department of Transportation, like many other transportation agencies, has
invested significantly in extensive closed circuit television (CCTV) systems to monitor freeways
in urban areas. Although these systems have proven very effective in supporting incident
management, they do not support the collection of quantitative measures of traffic conditions.
Rather, they simply provide a moveable platform for trained operators to collect images for
further interpretation. Although there are several video image vehicle detection systems
(VIVDS) on the market that have the capability to derive traffic measures from video imagery
automatically, these systems currently require the installation of fixed-position cameras. Thus,
they have not been integrated with the existing moveable CCTV cameras.
This research effort addressed VIVDS camera repositioning and calibration challenges
and developed a prototype machine-vision system that successfully integrates existing moveable
CCTV cameras with VIVDS. Results of testing the prototype in a laboratory setting
demonstrated that when the camera's original zoom level was at a level of Ix to 1.5x, the system
could return the camera to its original position with a repositioning accuracy of less than 0.03 to
0.1 degree. This is significantly less that the 0.5-degree accuracy of mechanical camera presets
and indicates that such an approach provides the accuracy needed for CCTV/VIVDS integration.
This level of positional accuracy, when combined with a VIVDS, resulted in vehicle count errors
of less than 1%. Based on these results, the integration of CCTV and VIVDS is feasible, thus
paving the way for less costly, more easily maintained traffic monitoring systems in future
intelligent transportation system initiatives.
111
FINAL CONTRACT REPORT
CAMERA POSITIONING AND CALIBRATION TECHNIQUES FOR INTEGRATING
TRAFFIC SURVEILLANCE VIDEO SYSTEMS WITH MACHINE-VISION VEHICLE
DETECTION DEVICES
Brian L. Smith, Ph.D.
Assistant Professor
Department of Civil Engineering
University of Virginia
Michael L. Pack
Graduate Research Assistant
INTRODUCTION
A fundamental function of a transportation management center (TMC), such as the Smart
Traffic Centers operated by the Virginia Department of Transportation (VDOT) in Northern
Virginia, Hampton Roads, and Richmond, is to monitor traffic conditions. Traditionally, two
independent subsystems have been used to support this function. First, vehicle presence sensors
were installed throughout the network to collect quantitative measures of traffic conditions,
including flow rates, average vehicle speeds, and sensor occupancy levels (a surrogate of traffic
density). These sensors are costly to install and maintain given the large number required and
the harsh conditions in which they operate. The second subsystem is a network of closed circuit
television (CCTV) cameras that are typically used by TMC operators to inspect traffic conditions
visually and to investigate details of traffic incidents to support improved response. As such, the
CCTV subsystem relies on moveable (i.e., pan, tilt, zoom) cameras. This subsystem has proven
to be particularly expensive to install given the high communications bandwidth requirements of
video transmission.
In Virginia, inductive loop detectors have traditionally been the presence sensors of
"choice" for TMCs. However, an attractive alternative that has emerged in the last decade is
video image vehicle detection systems (VIVDS). VIVDS use software to analyze digitized video
to identify the presence of vehicles in zones manually defined by engineers calibrating the
system. In other words, an engineer will install a video camera and define detection zones in
travel lanes, and then the software will essentially mimic the operation of an inductive loop
detector. The definition of the zones is a very important aspect that must be completed precisely.
Research has shown that if the camera is moved, it is quite difficult to reposition it adequately to
allow the VIVDS to continue operating using the originally defined zones (Cottrell, 1994). For
this reason, the accepted practice is to use VIVDS only in conjunction with fixed-position
cameras. This has prevented the integration ofVIVDS with CCTV systems.
Of course, the integration ofVIVDS with CCTV systems would provide the enormous
benefit of combining the infrastructure needed for the two key traffic monitoring functions of
TMCs. Simply put, the savings in terms of installation and maintenance costs that would be
realized are significant. For this reason, the Virginia Transportation Research Council proposed
this project, with the full endorsement and support ofVDOT's ITS Research Advisory
Committee.
To quantify this benefit conservatively, consider the impact of replacing a portion of the
existing loop detectors in the Hampton Roads Smart Traffic Center (HRSTC) with integrated
CCTV/VIVDS. The HRSTC currently includes 200 detector "stations," which generally consist
of loops in each of four travel lanes. To quantify the benefits, assume that integrated
CCTVIVIVDS will allow for the elimination of one fourth of the loops, or 200 loops total.
The costs of the current HRSTC monitoring subsystem, both in terms of initial
construction and annual maintenance, have been estimated based on unit costs published by the
USDOT's ITS Benefits and Unit Costs database (www.benefitcost.its.dot.gov). Table 1 presents
the estimated costs for the current HRSTC system, only considering the 200 loops in question.
Since most STCs in Virginia use large fiber-optic communications networks, communications
costs are not included in this analysis.
Table 2 presents the costs associated with HRSTC with a CCTV/VIVDS system
deployed, eliminating the need for loop detectors.
Thus, this conservative analysis indicates that VDOT could realize over $1 million in
construction cost savings for the HRSTC based on integrated CCTV/VIVDS deployment, as well
as a reduction in maintenance costs. In addition, the benefit would be even greater given that the
integrated CCTVIVIVDS system would not necessitate lane closures to repair detectors as is
currently necessitated by loops.
Table 1. Costs for Existing HRSTC Loops/CCTV
Component Magnitude
Unit
Price Constr
TotalConstruction
Unit
Price Maint
TotalMaintenance
Loops
200
6,500
1,300,000
650
130,000
CCTV
Towers
Processor
and Software
forCCTV
40
40
40,000
40000
1,600,000
1 600000
150,000
2,000
80,000
Integration
250000
4,900000
Total
2
20,000
230000
Table 2. Costs for HRSTC with Integrated CCTVNIVDS Deployed
Component Magnitude
TotalMaintenance
40,000
40,000
150,000
1,600,000
1,600000
150,000
3000
Processor
and
Software for
VIVDS
150,000
150,000
7,000
7,000
Integration
Total
250,000
250,000
3750,000
12,000
12,000
159,000
CCTV
Towers
Processor
and
Software for
CCTV
40
40
Unit PriceUnit PriceTotalConstr
Construction
Maint
120,000
20,000
PURPOSE AND SCOPE
The purpose of this research effort was to evaluate the technical difficulties associated
with integrating VIVDS with existing CCTV cameras and control equipment and seek to
determine the feasibility of developing automated, machine-vision techniques that effectively
address these difficulties.
The scope of this study was limited to prototype development and testing in a laboratory
environment designed to emulate a field installation. This minimized development and testing
cost and complexity as the feasibility ofCCTV/VIVDS integration was investigated.
METHODOLOGY
Four tasks guided the research effort:
1. Review the literature.
2. Design a prototype CCTV/VIVDS integrated system.
3. Implement the prototype.
4. Test the prototype.
3
Literature Review
A review of the literature was conducted to provide a foundation for the research. The
literature review focused on the three areas: (1) CCTV fundamentals, (2) VIVDS fundamentals,
and (3) CCTV/VIVDS integration.
Design of Prototype CCTVNIVDS Integrated System
Based on the lessons learned from the literature review and discussions with VDOT TMC
personnel, a prototype design was developed to integrate VIVDS with moveable CCTV cameras.
Implementation of Prototype
Using the design developed in Task 2, the prototype system was implemented in the
Smart Travel Laboratory. The system was built using the LabVIEW software package and its
associated image processing capabilities.
Testing of Prototype
The prototype system was tested in the laboratory to assess its potential for field
deployment. Two series of tests were performed: positioning and performance.
Positioning
The first series of tests were designed to determine the precision to which the prototype
system could reposition a camera in the pan/tilt/zoom fields. This set of tests was necessary to
ascertain if the repositioning was sufficiently accurate to allow for integration with a VIVDS
system.
A grid was created on a 4-ft by 4-ft screen. The grid was broken down into fractions of
an inch. Also drawn on the grid was a simple target pattern that could be tracked by the
prototype system. This screen was placed 25 ft away from a Pelco Esprit camera and integrated
prototype positioning system. A laser was then attached to the top of the camera with the beam
pointing in the same direction as the camera lens. Thus, the combination of the laser and grid
allowed for a precise determination of the camera's position. Two tests were run using the
experimental setup shown in Figure 1.
4
ROI to be tracked
by camera
\,
1--101--1--
~~
7
1
...
vI.-'
i-r-.,
~
~
.....
v
r--..
v
V
v
\
v
v
v
V
f\.
..
1\
Grid for measuring
repositioning error.
)
,
~~.
V •
I""'" v
vr-..,
I'..
r-..,
1
II
f'..
V
I'-.
~
v
f'..
i\.v
'" '"
I.....
v"\
.......
1\
\
'"
i'./
I/'
..... 1./
""""'I--.
l.-"L--"
Figure 1. Position Testing Experimental Setup
Test Case A: Zoom
Test A measured the precision of the prototype system when re-zooming the camera to a
desired level. First, the size of the grid when viewed on the monitor was measured, and then the
camera was zoomed out. Next, the prototype system attempted to re-zoom to the original zoom
setting. After the prototype system had finished, the size of the grid when viewed on the monitor
was measured. This change in grid size, or re-zooming error, was measured as the percent
difference from the original size
Test Case B: Pan and Tilt
Test B measured the prototype system's ability to reposition the camera in the pan and tilt
directions given that the zoom field had already been set. The prototype system was used to
locate the region of interest (ROI) drawn on the grid and position the camera to a set of
coordinates. Once the ROI had been tracked, the position of the laser beam hitting the screen
was marked as a reference point. Then the camera's pan and tilt settings were changed, and the
prototype system attempted to reposition the camera back to this reference point. This test was
repeated 10 times for each of four zoom factors.
Performance
Fundamentally, these tests measured the accuracy of vehicle counts from the VIVDS
system after the CCTV camera had been moved and repositioned by the prototype system.
Testing was conducted in a laboratory setting and was designed to emulate a real-world
freeway management system. A video recording of traffic was obtained from the Hampton
Roads Smart Traffic Center in southeastern Virginia. The recording was of a four-lane freeway,
5
(three standard lanes and one HOV lane) and was recorded from a CCTV camera mounted on
top ofa 60-ft tower. During taping, the camera's parameters were not changed (i.e., pan, tilt, and
zoom levels were held constant).
The laboratory set up, illustrated in Figure 2, consisted of a VCR, a video projector, a
large projection screen, two personal computers, the Pelco camera and positioning system, and
the Autoscope 2004 unit. The traffic video was projected onto a large, I5-ft by I5-ft screen. The
Pelco Esprit camera and integrated positioning system was positioned 25 ft away from the
screen. This distance was such that the video projected onto the screen matched the camera's
field of view in the field. Thus the images viewed on the Pelco video camera were of the same
scale as those recorded by the original TMC CCTV camera. One PC was connected to the Pelco
camera and was used for all image processing and camera control. A second PC was directly
interfaced with the Autoscope 2004 unit. After the Autoscope 2004 unit was calibrated, it
digitized video frames of traffic from the Pelco camera, processed them, and measured vehicular
traffic counts.
Four tests were conducted to evaluate the VIVDS traffic measurement performance using
the prototype system:
1. normal field of view during daylight (zoomed at Ix)
2. smaller than normal field of view during daylight (zoomed at I.5x)
3. smaller than normal field of view during daylight (zoomed at 2.0x)
4. normal field of view at nighttime (zoomed at Ix zoom).
Testing was conducted only at these zoom levels because further zooming would
typically eliminate full viewing of the road. Zooming the camera further than this is usually
reserved for specific incident inspection where a TMC operator may need to inspect a single lane
of traffic very closely to verify the nature of an incident. TMC operators generally prefer to have
the CCTV camera zoomed out as far as possible under "normal" conditions so that the amount of
freeway they can monitor is maximized.
Each of the four tests evaluated the prototype system performance against two other
methods of repositioning: (1) manual repositioning, and (2) camera "preset" repositioning. The
tests were run using the following procedure:
6
......,J
~
"'C
=
l'"'to-
t'D
rJJ.
l'"'to-
=
e:..
t'D
t'D
.,
S-
"'C
~
~
~
5-
l'"'to-
{IJ
t'D
~
t'D
~
=
~
8
.,~
.,
~
t'D
rJJ.
~
<
<
~
~
t'D
.,=
~.....
-.- ....
--- ..... - ....
-. .....
--- ............ _-
PrOjector
PC2 for Autoscope
2004 cahbration and
data coUe cnon
CO.A:X. cable for Vldeo feed
1.
Autoscope 2004
~~;:
Petco
camera with
VCR
-......-::::----------',
.
,..,.../"'-----.....
RS -4 22 for PTZ control
----~_._-----------------
PCI for AUTOTRACK
pattern matchmg and PTZ
control
---
/
1. The camera was first aimed at the projection screen and the desired pan, tilt, and
zoom (PTZ) fields were set. This PTZ position was programmed into the camera as a
"preset."
2. An appropriate ROJ was selected. For all tests, the ROJ used for pattern matching
was the HOV diamond already drawn in the HOV lane. Although this is not an ideal
ROJ, it did have sufficient contrast and was large enough that it could be detected by
the pattern matching algorithms.
3. The Autoscope 2004 detection zones were calibrated to the view from this current
camera configuration.
4. The traffic videotape was played and traffic volume data were collected with the
Autoscope unit.
5. After this "base" dataset was collected, the camera was moved out of position such
that the detector zones were no longer in the correct place but the ROJ was still
visible.
6. The prototype system was then used to automatically reposition the camera.
7. The same 3D-minute segment of traffic video was played, and data were again
collected with the Autoscope using the same detector configuration.
8. After this second data set was collected, the camera was again moved out of position.
9. An operator was allowed to try and reposition the camera to its originally calibrated
position using a simple joystick, similar to those used in TMCs for repositioning
cameras.
10. The same 3D-min segment of traffic video was played and data were collected with
the Autoscope unit using the same detector configuration.
11. The camera was again moved out of position one last time.
12. A command was sent to the camera to return to the "preset" position that had been
programmed in step 1.
13. The same 3D-min segment of traffic video was played and data were again collected
with the Autoscope 2004 using the same detector configuration.
8
RESULTS
Literature Review
Key findings of the literature review are presented in the areas ofCCTV fundamentals,
VIVDS fundamentals, and CCTVIVNDS Integration.
CCTV Fundamentals
TMC operators use CCTV cameras primarily to visually detect and determine the cause
of freeway incidents (Federal Highway Administration, 1997a). These cameras are almost
always mounted on pan/tilt controllers fixed high atop some type of tower sitting next to the
freeway or in the median. The pan/tilt controllers combined with the height of the towers allow
TMC personnel to monitor large sections of the freeway should an incident occur. Although
most TMCs usually leave the cameras in a fixed position during "normal" conditions, it becomes
necessary to be able to PTZ the camera at various levels to inspect or verify an incident.
VIVDS Fundamentals
VIVDS incorporate image processing techniques to analyze frames of video to determine
the presence of a vehicle at a specific point on a roadway. All VIVDS require an image sensor
(camera) to acquire an image. In addition, VIVDS use a digitizer to convert the analog video
signal to a digital image and a microprocessor plus software for real-time video image analysis
and traffic parameter extraction. Advanced machine-vision, pattern-recognition algorithms are
used to detect vehicles under various environmental and traffic conditions. The detectors
(generally count and speed detectors) are generated and configured as overlays on a video
monitor through interactive graphics using a PC and mouse (Cottrell, 1994).
Though VIVDS are generally good at collecting traffic data, they can do so only if they
are calibrated properly and the camera is positioned correctly. VIVDS are very sensitive to their
calibration parameters. In particular, since the detectors are "virtually" defined on a PC monitor
and are calibrated to the acquired image, it is imperative that the camera is not moved in either
the pan, tilt, or zoom fields. Doing so would mean that the virtual detectors would no longer be
in the "correct" place, and the VIVDS would attempt to identify vehicles outside the normal
travel lanes, resulting in many missed and false detections.
CCTVNIVDS Integration
To date, VIVDS have not been successfully integrated with existing moveable CCTV
systems. This is primarily due to the fact that VIVDS require a fixed image sensor. As
concluded in a 1997 study: "Even slight movement can misalign [detection] zones" (Federal
Highway Administration, 1997b). For example, Figure 2 demonstrates how an inability to
9
reposition a camera's pan and tilt fields properly will result in "misplacement" of detection
zones.
Several studies have addressed the issue of integrating VIVDS with moveable CCTV
systems (Cottrell, 1994; Namkoong et aI., 2000). Three basic strategies have been considered:
(1) repositioning CCTV cameras to realign with original VIVDS detection zones; (2)
automatically recalibrating the detection zones using edge detection and image difference
methods to determine where the roadway and each individual lane are positioned; (3) analyzing
panning, tilting, and zooming factors of the CCTV cameras to allow for continuously
recalibrated detector zones as a camera is repositioned. Finally, it is important to note the
difference between "recalibration" and "repositioning" techniques. Recalibration refers to the
process of redefining where the VIVDS' s virtual detectors are placed on the screen.
Repositioning refers to the realigning, or re-aiming, the CCTV camera such as it was positioned
when the VIVDS detectors were originally calibrated. In other words, repositioning means that
VIVDS "recalibration" does not have to occur and vice versa.
Researchers have met with little success using these strategies. VDOT attempted VIVDS
CCTV integration via manual repositioning in 1994 with no success (Cottrell, 1994). It was
concluded that operators could not reposition cameras accurately enough to maintain VIVDS
effectiveness. Researchers in Korea have attempted to recalibrate the virtual detectors of a
VIVDS automatically each time the camera is moved using edge detection techniques
(Namkoong et aI., 2000). However, this methodology met with limited success because edge
detection was relatively ineffective, especially at night. Finally, automatic recalibration of
VIVDS suffers from the fact that the VIVDS will be constantly recalibrating itself, even when
the camera is awkwardly positioned to investigate an incident, ultimately resulting in poor
detection performance.
Given the sensitivity ofVIVDS performance to the quality of initial calibration, it is
conceptually advantageous to have a professional set up and calibrate a VIVDS so that all of the
angles and zoom factors are correct and then attempt to reposition the camera precisely each time
the camera is moved. In this research, automated rather than manual repositioning (as in the
1994 VDOT study) was investigated. Though many PTZ controllers now have the ability to
reposition cameras using programmable preset positions, these "presets" are accurate only to
within 12 of a degree. This degree of precision when a camera is 400 ft from the road (typical for
TMC CCTV systems) results in a sensor calibration error of nearly 4 ft. This would be
unacceptable for any VIVDS as the detection zones would constantly be improperly
repositioned. Thus, there is a need for an alternative repositioning approach with greater
precision. In this research, an automated repositioning methodology was developed using
machine-vision technology. The system incorporates normalized cross-correlation and patternmatching techniques to search for a "target" placed near the travel lanes (such as in the median).
The design of the prototype system is described in the next section.
10
Prototype CCTVNIVDS Integrated System
The prototype system, referred to as Autotrack, was developed using the image
processing capabilities of the LabVIEW software package. Autotrack incorporates two main
phases: (1) a learning and calibration phase, and (2) a pattern matching and camera PTZ control
phase.
Figure 3 is a flowchart of the first phase of the process. This phase can be described as a
six-step, non-iterative process. Steps 1 and 2 of Phase 1 are performed using the Autotrack
algorithm implemented in LabVIEW. Step 3 is performed with the VIVDS system. In step 4,
the user selects a ROI on the PC screen using a mouse. In this case, the ROI is the target zone
that has been located on the median. This ROI is automatically captured and saved in steps 5
and 6 as a template used in the pattern matching function of Phase 2.
1. Acquire
Image
4. Select
ROJ (target)
..........
5. Learn the
ROJ
j~
~r
2. Position camera
forVIVDS
calibration
~,
....
JIll""'"
3. Calibrate
detection zones
in the VIVDS
6. Learn the
ROJ's
coordinates
Figure 3. Autotrack Algorithm Phase 1: Learning and Calibration
The second phase consists of the pattern matching and camera control functions. This
phase can be broken down into two iterative processes: the zoom repositioning process (Process
I), and the pan/tilt repositioning process (Process II). Figure 4 is a flowchart depicting these
processes.
The following subsections provide more detail on key aspects of the Autotrack algorithm:
pattern matching, scale variation, and pan/tilt control.
11
r--------------------------------------------------------------------------------,
1
1 ...---------.
:
:
I
H1stogram
match1ng bas ed
Zoam camera
all the way
out
S e3fch 1mage
forROI
nn t1mp. nfrirtv
IsROI
found?
NO
YES
Process I
Is score
lugherthan
preVlous
YES
~ ..... nf"P?
L
,
1
I
r--------------------------------------------------------------1
1
H1stogram
matclungbased
on t!me of day
Se3fch 1mage ~
forROI
.
I
Return
co onunates
of the ROI
: ...----""""""'------.
.....
I
1-4__- -
T11t camera 0 ne step
1n the direct! on 0 f
the deSlred
co ordlnates.
Process II
Do the coordlnates
match the desired
co ordinates?
Pan camera one step
1n the dlre ct1 on 0 f
the des1red
co ordlnates
NO
YES
Figure 4. Autotrack Algorithm Phase 2: Pattern Matching and Camera Control
12
Pattern Matching
Pattern matching is the technique used to locate known reference patterns in an image
quickly. Normalized cross correlation is an effective technique for finding patterns in an image,
as long as the patterns in the image are of the same size (i.e., they are not scaled) and not rotated.
Typically, cross correlation can detect a pattern of the same size up to a rotation of 5 to 10
degrees.
The pattern-matching algorithm used in Autotrack returns a list of matches, the position
of each match, and a score indicating the closeness of each match. The number of matches, in
this case, is not a concern since only one ROI is being searched for. The returned coordinates
will, however, be extremely important. Further, the "score" will also be extremely important
later when searching for scale variations.
The pattern-matching algorithms used in LabVIEW's IMAQ VISION software are fairly
robust. They tend to work well even when lighting is poor, some blur is present, or the image is
noisy; however, the pattern-matching algorithms compensate only minimally for changes in
scale. It will return matches with scale variations of up to about 5%. For this particular
application, changes in scale would be due only to changes in the zoom parameter of the CCTV
camera. As the camera zooms in or out, the ROI will appear to either grow or shrink in size. The
challenge then was how to locate the ROI even when the camera was zoomed in or out such that
the ROI was no longer detectable due to changes in its scale.
Scale Variation
Camera zoom control is adjusted until the ROI is detectable by the normalized crosscorrelation procedure. Once the ROI is located, the camera continues to zoom as the pattern
matching score returned by the algorithm increases. As soon as the score begins to decrease, the
zoom backs out again to the last, highest score. This is then considered to be the proper zoom
level.
Pan and Tilt Control
When the ROI is detected and coordinates are received, the Autotrack algorithm sends
move-stop commands to the pan/tilt unit to reposition to the original ROI coordinates.
Prototype Implementation
The following procedure functionally describes how Autotrack works:
1. The camera's PTZ fields are adjusted by an operator for the VIVDS traffic data
collection calibration.
13
2. The ROI is located on the screen, and a box is drawn around it using the mouse. In
the example provided in Figure 5, note that an HOV "diamond" serves as a very
effective ROI.
3. The "Learn" button is pressed, and an automatic template is created that stores
information about the ROI for tracking purposes.
4. VIVDS (in this case Autoscope 2004 by Econolite) virtual detectors are then defined
and calibrated (as demonstrated visually in Figure 6), allowing traffic data collection
to begin.
5. Once the camera is moved out of the position at which the VIVDS was calibrated
(such as when an operator pans, tilts, and zooms to investigate an incident), the
"Track" button is pressed. The Autotrack program then locates the ROI in the field of
view of the camera and repositions the camera in the PTZ fields.
HOV Dianl0nd
Figure 5. Locating an ROI
Figure 6: VIVDS Virtual Detectors
14
Testing
As described in the methodology section, two series of tests were conducted to evaluate
CCTV positioning and VIVDS performance. This section presents the results from this testing.
CCTV Positioning
Two test cases were considered for the CCTV positioning series of tests. Test A
examined the effectiveness of automatic zoom control, and Test B considered pan and tilt
control.
Test Case A: Zoom
The results of this test are presented in Figure 7. As seen in these results, zoom control
accuracy is non-linear and difficult to quantify. Re-zooming to a low zoom level is nearly 100%
accurate; however, when repositioning to a higher zoom level, 3x for example, an error of 5% is
not uncommon.
6
5
...
......
w
0
...r::
••
4
• •
•
•
•
••
•
•
••
3
...~
G)
c.. 2
.
•
•
•
•
••
1
0
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
Zoom Factor
Figure 7. Results of Test Case A: Zoom Control Error
Test Case B: Pan and Tilt
The displacement errors for each repositioning attempt were measured and are presented
in Table 3. Note that the average error, measured in degrees, never rises above ~ degree of
precision (as opposed to the Y2 degree of precision offered by mechanical presets). As was stated
earlier, most camera positioning systems with "preset" programmability operate with a
repositioning accuracy of 12 degree of precision. This indicates that the Autotrack algorithm can
reposition the camera more accurately than most camera PTZ positioning preset systems.
15
Table 3. Pan/tilt Displacement Error by Zoom Level
Run
,1.0x
ZOom Level
1.25x - '1.5x -
1.75x
2.0x -
1
2
3
4
5
6
7
8
9
10
0.05
0.00
0.03
0.05
0.00
0.00
0.03
0.03
0.00
0.03
0.04
0.00
0.11
0.13
0.00
0.06
0.00
0.05
0.00
0.10
0.00
0.15
0.15
0.15
0.00
0.00
0.00
0.15
0.00
0.14
0.00
0.17
0.19
0.06
0.00
0.00
0.19
0.11
0.05
0.00
0.05
0.07
0.00
0.00
0.23
0.00
0.21
0.10
0.00
0.21
p
0.02
0.02
0.05
0.05
0.08
0.08
0.08
0.08
0.09
0.10
tT
(I
e
CD
.,
Q
--.e
c
UJ
VIVDS Performance
The results from the four tests to evaluate the VIVDS traffic measurement performance
using the prototype system are presented here.
Normal Field of View at Day (lx Zoom Level)
The first test was conducted with the camera at a zoom level of 1x. The results presented
in Table 4 are the vehicle count average of three runs of the test procedure. The count error
percentage associated with the Autotrack's performance, seen in Figure 8, is negligible for this
run and is thus difficult to see on the graph. In all instances, the count error with the Autotrack
system was less than 1%. Note that the error when manual repositioning is used is significantly
higher than both the camera's "preset" repositioning and the Autotrack repositioning. The
highest error observed, 38%, occurred using the manual repositioning approach in the HOV lane.
Table 4. Count Results for Day Testing at Ix Zoom Level
Vehicle Count
Lane
HOV
Left
Middle
Right
Initial
Calibration
AUTOTRACK
Manual
' Preset
152
653
811
705
152
654
813
708
94
629
912
845
148
660
825
700
16
40.00
35.00
30.00
....c:
25.00
, ........... , ......... ,
.... "'-....
CD
....0 20.00
,,,
o Manual
..................................
• Preset
CD
D.
• AUTOTRACK
15.00
10.00
........................
",
"
............... ,,'"
5.00
,
, ........ ", ....
....
.....
, ..... , ..... "
"" ............
0.00
HOV
Left
Right
Middle
Lane
Figure 8. Percent Error for Day Repositioning Tests: Ix Zoom. Autotrack error is essentially 0 in the left,
middle, and HOV lanes.
Smaller Than Normal Field of View at Day (Zoomed 1.5x)
The second test was conducted with the camera at a zoom level of 1.5x. The count data
presented in Table 5 are an average of three runs of the test procedure. The count error
percentage, presented in Figure 9, associated with the Autotrack's performance is again
consistently less than 1%. The Autotrack repositioning method continues to outperform the
manual and "preset" positioning methods. However, because the repositioning accuracy in the
zoom level is decreased as the zoom level increases, more error is present than in the previous
test.
Table 5. Count Results for Day Testing at 1.5x Zoom Level
Vehicle Count '
lane
Initial
Calibration
HOV
Left
Middle
Right
150
650
815
708
AUTOTRACK
' Manual
Preset
151
652
818
712
138
670
840
706
148
658
825
715
17
9
8
7
• AUTOTRACK
6
EJ Manual
• Preset
3
2
1
o
HOV
LEFT
Middle
Right
Lane
Figure 9. Percent Error for Day Repositioning Tests (1.5x Zoom)
Smaller Than Normal Field of View During Daylight (Zoomed at 2. Ox)
The third test was conducted with the camera "starting" at a zoom level of 2x. The
maximum percent error associated with the Autotrack repositioning method increased to 3.7% in
the HOV lane when the zoom level was increased to this level. Full details of the evaluations
from this test are available elsewhere (Pack, 2002). This increase in error at higher zoom levels,
as seen in the positioning tests, indicates that the prototype system should be used only at low
zoom levels.
Normal Field of View at Nighttime (lx Zoom)
The performance of the Autotrack program at night compared to the performance of the
two other repositioning methods was nearly identical with the performance of the Autotrack
system during the day in terms of the percent error. The largest error for this run was in the
right-most lane of traffic and was still under ~ of 1%.
Factors Affecting Performance
Although the prototype Autotrack algorithm demonstrated an excellent performance, a
number of reasons indicate that the results presented are likely those of a "worst case"
implementation for the following reasons:
18
•
The Pelco Esprit camera used for testing is a high-speed camera. That is, the speed at
which the zoom lens was rotated was quite fast. Thus, it proved difficult for the
Autotrack algorithm to issue a stop command fast enough in the zoom repositioning
process. This resulted in zoom "overshooting" errors. Many cameras on the market
provide the ability to control the speed at which they zoom. Such cameras would be
better suited for this application. Further, older cameras that zoom more slowly could
also work better.
•
The video images that were processed using the prototype system were a digitized
video projection of a VHS recording being displayed onto a screen. This adds
degradation layer upon degradation layer to the quality of the video. This combined
degradation limits the system performance.
CONCLUSION
This research investigated the premise that existing TMC CCTV cameras and equipment
could be used effectively in conjunction with VIVDS. Integrating CCTV with VIVDS presents
challenges because VIVDS traditionally require a fixed-position camera to operate properly and
TMC CCTV cameras are free to be moved in the PTZ directions.
The research team developed a machine vision-based automatic recalibration procedure
that enables the integration ofVIVDS/CCTV. This was demonstrated with a prototype system
that supported nearly error-free volume data collection (on the order of 1% or less at reasonable
initial camera zoom levels) in a laboratory test. This is a significant conclusion that contradicts
the prevailing opinion in both research and practice. The conclusion that VIVDS/CCTV is
feasible paves the way for significant costs savings in ITS system deployment and maintenance.
RECOMMENDATIONS
1. Develop a modified prototype Autotrack system that is suitable for field deployment with
existing CCTV cameras. The field-level Autotrack prototype should be evaluated extensively
to make a final decision regarding CCTV/VIVDS integration in future and
redesigned/redeveloped transportation management systems.
2. When purchasing new or replacement CCTV hardware during the field testing period,
procure cameras that support software-based PTZ control. In addition, cameras should be
purchased that allow for software-based control of zoom speed. This will support future
implementation of a field version of the Autotrack system.
19
REFERENCES
Cottrell, B.H. Evaluation ofa Video Image Detection System. VTRC 94-R22. Virginia
Transportation Research Council, Charlottesville, 1994.
Federal Highway Administration. Freeway Management Handbook. FHWA-SA-97-064.
Washington, DC, 1997a.
Namkoong, S., Lee, I.J., Min, J.Y., and Yun, B.J. An Algorithm for Automatic Installation of
Detection Area by Analyzing the Panning, Tilting and Zooming Factors ofCCTV
Camera. Proceedings of the 7th World Congress on Intelligent Transport Systems. Turin,
Italy. Published by ERTICO, Brussels, Belgium. 2000.
Pack, M.L. Automatic Camera Repositioning Techniques for Integrating CCTV Traffic
Surveillance Systems With Video Image Vehicle Detection Systems. Master's Thesis in
Engineering. University of Virginia, Charlottesville, 2002.
20
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement