Applications of Video Surveillance Systems for Traffic Data Acquisition

Applications of Video Surveillance Systems for Traffic Data Acquisition
Applications of Video Surveillance Systems for Traffic
Data Acquisition
Vasil Lakov
University of Architecture, Civil Engineering and Geodesy
1046 Sofia, Bulgaria
[email protected]
http://www.uacg.bg/
Abstract. The process of urbanization raises city population and extends city
boundaries, which leads to heavier road traffic, environmental pollution and
bigger financial losses. Due to spatial structure and influence of many factors
on transport conditions, the transport system requires complex management
methods. To be widely adopted, traffic management projects have to have the
lowest possible price and highest effect on transportation. To accomplish these
requirements, city authorities have to know what and where the halting points
are. Using existing CCTV cameras for traffic purposes can empower authorities to easily obtain city-wide traffic data. This paper analyzes the benefits and
problem areas of CCTV systems when used for traffic surveillance and present
different ways for improving reliability and accuracy of these systems. Examined areas are camera settings, optical filters, mounting place, digital image
processing techniques for image enhancing and object extraction and objects
filtering based on their properties.
1 Introduction
The rising of the world population and the process of urbanization are some of the
reasons for differentiation of densely populated city areas. The growth of cities in
both horizontal and vertical direction, strengthening of economic links among them
and constant searching for profit require more effective ways for people and goods
transportation. Road transport arteries connecting residential with industrial and
commercial city areas, and highways between cities, gather extremely important role
in functioning and growth of every modern country.
Mass usage of automobile transport is constantly increasing since its biggest advantage is door-to-door service. To be able to guarantee normal functioning of the
global economical system, road transport systems must be:
 efficient (providing alternative routes, enough capacity, suitable for all vehicles),
 effective (with minimal expense of time and fuel),
 reliable (with minimal disturbance from planned and random events),
 ecological (with minimal environmental pollution),
 safe (with minimal risk for traffic participants).
As reasons for traffic jams, and the ways of reducing them are complex. The ways
for improving transport quality may be grouped in the following categories:
 restricting or stimulation of using one or another type of vehicles and traffic reorganization,
 city areas and roads planning consistent with current and future transport needs,
 renewal of old one and constructing of new road infrastructure,
 enhancing of traffic flows surveillance and control.
2 The role of traffic sensors
Intelligent Transportation Systems (ITS) are in use worldwide nowadays and their
aim is to improve safety, performance, environmental impact and to provide sustainable development of road transport systems by modern transport technologies.
For identifying the bottlenecks and their locations, as also for revealing and evaluating the possible corrective actions, it is mandatory to have information about the
road infrastructure and the traffic flows. The realization of a specific traffic project
can influence different aspects of transport conditions and evaluating the project
consequences in advance is an important step in its planning. By creating computer
models of city or national road networks and by using real stress data acquired by
traffic sensors optimal decisions are more easily taken in various situations during
traffic management and planning in response to current demand and conditions
[1,2].
Traffic flow management relies on up-to-date data for different traffic parameters.
In addition to individual vehicle parameters sensors compute generalized indicators
describing automobile flow as a whole. This data is used for a wide range of tasks
such as:
 evaluation of road load,
 simulation of computer transport models,
 adaptive control of traffic lights,
 identifying of peek hours,
 identifying locations with frequent traffic congestions,
 evaluation of sound and atmospheric pollution,
 determination of the predominant types of vehicles,
 route determination of major traffic flows.
Examples of measured vehicle parameters are:
 used lane,
 direction,
 speed,
 type of vehicle,
 weight of the vehicle,
 length of the vehicle,
 time of passage,
 registration number,
On the basis of these parameters following indicators can be computed:
 average flow speed,
 distance between consecutive vehicles,
 loading degree of the road section,
 air and noise pollution,
 existing or impending congestion,
 traffic violations.
For the realization of the most suitable project meeting expected improvement of
transportation conditions it is necessary to clarify beforehand the following questions:
 what is the transport problem,
 what information is needed for solving the problem,
 what are the specifics of the road section and the traffic.
Having defined the above points a project may proceed with the choice of technology
and specific road sensors meeting the requirements.
3 Traffic sensors
The two main groups of traffic sensors are:
 in-vehicle sensors,
 road sensors.
The first group of sensors represents devices such as mobile phones, GPS receivers
and RFID transponders, mounted in each vehicle and monitoring only this vehicle.
In this case, many devices are used for ensuring proper data and covering larger area
of the road network.
The second group of sensors is installed at a specific point of the network for
measuring local traffic parameters only. One road sensor can monitor multiple lanes
with the same or different directions. This group includes video surveillance systems
for acquiring and analyzing traffic data. These are passive sensors covering long
road section, providing simultaneously video signal from the place, which is the
major difference compared to other types of road sensors. For extracting useful traffic
data from the video signal different algorithms for video image processing are being
applied [2].
4 Video road sensors
Video road sensors are based on video surveillance systems and techniques from
machine (computer) vision for analyzing of video frames in the process of acquiring
data for the vehicles visible in the frames. The video camera frames can be processed
either centralized in the control centre or locally by each video camera.
In the first case, computers are located in the data centre and are processing video
streams from remote cameras, as one computer or controller can handle multiple
video streams. In the second case, video cameras are equipped with integrated computer modules for digital image processing. These video cameras transmit only traffic data to the control centre, but it is also possible to stream live video. The choice of
the video system type depends on various factors, whereas it is possible to establish a
hybrid system including already installed analogue cameras as well as new smart
digital cameras. Streaming of the video signal from the cameras can be permanently
or when certain conditions are being registered such as abnormal parameter values, a
traffic incident or a violation [3].
Video road sensors implement two types of algorithms for measuring vehicle parameters. The first one measures parameters only once in a set point of the video
frame, normally when vehicle crosses a virtual line, thus simulating physical road
sensors. The other type measure constantly vehicle parameters, as long as vehicles
are visible. This process includes tracking each vehicle on each consecutive video
frame.
In order to be appropriate for road traffic monitoring, video road sensors must fulfil following requirements:
 automatic detection of all vehicles,
 detection of both moving and stopped vehicles,
 operability under various weather conditions,
 operability in real time.
To be considered a universal tool in transportation projects video road sensors must
also provide:
 high accuracy of the measured parameters,
 simplicity of installation and maintenance,
 high fault tolerance,
 ability to exchange information with other systems,
 ability to visualize and analyze collected data.
5 Advantages of video sensors
In the big cities, there are already established systems for video surveillance of various public places, including important road arteries for the purpose of security. The
signal from these video cameras can be used simultaneously for the purpose of traffic
monitoring and safety. In this case the whole infrastructure is ready and necessary
investments are substantially lower. What remains to be done is the installation of
computer equipment in data centres for processing video signals from cameras. This
significantly reduces the time for building the system.
With the ability to cover a large number of lanes and a long road section by one
video sensor different road situations can be detected e.g.:
 speeding,
 stopped or parked vehicles,






driving in forbidden lane,
driving in wrong direction,
crossing a continuous line,
red light crossing,
smoke in tunnels,
debris on the road.
This allows obtaining extensive information about the traffic by installing a small
number of devices. Installation of new cameras can be done on the side of the road,
as not to hinder normal traffic.
Adding new features to the processing of the video signal can be done in stages,
starting with vehicle counting and completing the system with their classification
and violations and congestion monitoring. The modular structure of the software
system will further accelerate the introduction of the system in operation and will
allow the creation of optimized solutions for every specific project. The general use
of video cameras eliminates the need to install other cameras for obtaining video
images used by supervisory authorities to assess the traffic situation.
Thanks to the digital nature of the video sensor functioning and modern communications, it is possible to analyze video signals from anywhere in the world. To
reduce needed bandwidth of the communication system, it is advisable to send first
only a picture and after the situation has been assessed by operator to begin video
broadcast in real time.
6 Disadvantages of video sensors
Difficulties in the operation of video sensors are a consequence of the difficulty in
extracting the necessary data from the video footage in degraded frame quality,
caused by the following factors:
 low light and bad weather conditions,
 low quality of video frames,
 scene complexity.
6.1
Low light and bad weather conditions
Insufficient light on the road is a major problem for video sensors. Separating cars
from the background becomes unreliable, which leads to missed cars. Many algorithms for detecting cars do not work as well at night.
Bad weather such as heavy rain, snow and fog reduces the road lighting and image contrast and may introduce noise in the frame. Visibility is also reduced and the
recognition of distant vehicles becomes difficult. The algorithms which use specific
points or object edges for identifying and tracking of vehicles will not be able to
reliably detect vehicles due to reduced contrast. Reduced number of the contours in
the image as a result of decreased contrast is used by some systems for smoke detection in tunnels.
6.2
Low quality of video frames
The quality of video frames is influenced by the following video camera elements:
 camera lens,
 image sensor type,
 automatic video functions.
Optical distortions are defects caused by a simple single lens or other imperfect optical system due to some phenomena and geometrical characteristics of light passing
through each lens. Each lens suffers, to some extend, from various optical defects. As
a result of these effects, the image created by the optical system is poorly focused,
distorted or with changed colours. Distortions are of several types and are removed
by a combination of different lenses. Completely removal of all distortions is impossible, and only these distortions that hinder the specific task are removed. This is
done through complex combinations of lenses of different shapes, made of glass with
different refraction ratios and combined in a way so that to remove all undesired
image faults. Compensating the various distortions is necessary for the proper measurement of vehicle speed, trajectory determination and classification [4].
A smaller photosensitive element creates a stronger noise in the image and if the
frame resolution is also low, the details in the image become indistinguishable,
which worsens the overall accuracy of the extracted data.
CCD and CMOS sensors represent two different technologies for capturing digital
images. Each has its strengths and weaknesses depending on the application and
neither of the two is clearly superior to the other. For the purpose of traffic monitoring the frame speed is the one of the standard TV and the time for digitalization of
the frame is not so important. But it is important to minimize the noise, keep the
colour uniformity and light sensitivity at high levels because traffic cameras work
24/7. Characteristics of both technologies are shown in table 1.
Table 1. Characteristics of image sensors.
Indicator
Chip output
Number of amplifiers
Noise level
Complexity
Dynamic range
Colour uniformity
Conversion speed
Power consumption
Heating
Pixels density
CCD
Voltage (analogue)
One
Low
Low
High
High
Medium to high
High
High
High
CMOS
Bits (digital)
For each pixel
Medium
High
Medium
Low to medium
Higher
Low
Low
Medium
CCTV cameras have a set of functions for automatic determination of parameters for
frame capturing depending on ambient conditions. For road sections monitoring,
changes in the scene are small and slow, and the most important factor is the total
light during the day. Some functions are not needed and removing or disabling them
facilitates the processing and reduces the cost of the camera. These unwanted functions are BLC (Back-Light Compensation), AWB (Automatic White Balance), AF
(Automatic Focus) and AI (Automatic Iris). In the absence of light sources behind
the road BLC should be blocked. The colour change with AWB can reduce the reliability of the algorithms using colour information for detecting vehicles. AF and AI
functions apply mechanical techniques which reduce the reliability of the camera and
increase the complexity and cost. After mounting the camera the focus remains unchanged and the AI function is replaced by optical filters, AGC and AES.
6.3
Scene complexity
When the video camera is mounted close to the ground or directed almost perpendicular to the road, it is possible for nearby vehicles to get overlapped and even hidden one by another. This leads to wrong counts or incorrect parameter values of
these cars. Merging of close cars is possible also during periods when the sun is close
to the horizon. Then cast shadows are longer and may fall on adjacent cars [5].
The presence of moving objects near the roadway or of ones often passing through
it can also lead to the adoption of these objects by the processing algorithms as real
cars. In this case, false alarms of traffic violations or other dangerous situations are
more likely to be generated by the system. Such unwanted objects can be:
 wild animals,
 pedestrians,
 moving trees,
 buildings.
Trees located adjacent to the road may hide part of it or be source of movement in
the frame (moving cars) in the presence of wind. Light sources at night can blind the
camera, illuminate excessively part of the road or be recognized as car headlights.
Glare from road surface (especially when wet) or vehicles surfaces in some cases can
be the cause for the impossibility to determine the vehicle type, the fusion of cars and
road and the loss of colour information. Even if these unwanted objects are out of the
analyzed area of the frame, they can change various parameters used in frame processing, such as image thresholding and objects filtering values.
During observation of roundabouts and squares with divers streams of cars and
pedestrians, where bus stops are present together with traffic lights, recognition and
filtering of the various objects and determination of their behaviour is very difficult.
If the road is curved and the system checks for violations as prohibited overtaking or
driving in prohibited lane, the vehicle trajectory will be an arc and not straight line,
which complicates its analysis.
7 Hardware methods for improvements
In creating a universal tool for traffic monitoring for all tasks and conditions, with
regard to the accuracy and reliability of the final results, it is appropriate to overcome the problems at each stage. The optimization process has to start with choosing
the camera and its location, to continue with setting camera parameters and to close
with video footage processing and filtrating algorithms and parameters.
7.1
Mounting place and video camera parameters
The cameras are mounted on a certain height above or on the side off the road. They
have to be pointed down towards the road, and all observed lanes have to leave the
frame at the top and bottom, not sideways. The sky must not enter the frame in order
to reduce ultraviolet light, brightness changes, camera dazzle by headlights, etc.
Surrounding objects and light sources, which may hinder processing algorithms,
must also be minimized.
Stationary cameras without zoom capability and progressive frame scanning are
used. Interlaced scan cameras require further signal processing before information
from the signal can be extracted.
To avoid having to adjust the iris, lenses with shorter focal length have to be used
to keep the depth of field long enough during the night, when the iris opens to allow
more light. A wide-angle lens provides a larger viewing area, but it introduces also
heavier geometric distortions.
When expected that the substructure will experience vibrations, it is necessary to
use a camera with image stabilization. It is sufficient to compensate the linear displacements of the image sensor, because there is usually no rotation. Stabilization
can be achieved in two ways – mechanical, by movable lens and digital, by image
processing.
Useful video functions are AGC (Automatic Gain Control) and AES (Automatic
Electronic Shutter). AGC function adjusts the gain of analogue signal from image
sensor at different levels of illumination. This function is useful during the evenings
and night. AES function is responsible for the shutter speed, which affects the
brightness of the frame. For many algorithms is important that the difference between consecutive frames is minimal. To achieve this, the two functions should
work, so that the parameters of the frame remain constant, even when the amount of
sunlight changes. Rapid change in brightness of the image is observed when a large
vehicle passes near the camera. Such a vehicle occupies big part of the frame and the
large surface reflects more light toward the camera.
7.2
Additional optical filters
The overall quality of the video frames can be improved by adding optical filters in
front of the camera lens. Filters for contrast enhancement are suitable for places with
frequent fog or haze. Polarizing filters reduce the glare from water, wet and metal
surfaces such as the coverage of most vehicles. Ultraviolet and infrared light filters
improve colour reproduction and durability of the image sensor in the video camera.
7.3
Processing units
The performance of microprocessors is growing steadily and today it is sufficient for
the purpose of digital image processing in real time and in particular for video traffic
monitoring. There are different technologies for the construction of computer modules for general and specialized applications, such as:
 digital signal processors (DSP),
 system on a chip (SoC),
 field-programmable gate arrays (FPGA),
 application-specific integrated circuit (ASIC).
Some of the solutions use hard-coded algorithms that can not be changed once the
chips are manufactured or once software is written to the chip. Since all functions of
video traffic sensors are realized by a software program it is desirable to be able to
change and enhance them after initial system deployment. In this sense it is better to
use standard solutions such as DSP processors, enabling easy software update.
When using a dual-core DSP processor, each core can process an individual
frame, so the cores will have twice the time to finish the processing. Another option
for optimization is to separate the series of processing operations between the two
cores. So the volume of tasks for each core will be smaller and will require less time.
8 Software methods for improvements
Having selected the optimal location and camera, here comes the optimization of the
filtering and processing algorithms for video signal analysis. Filtering is a basic
method for removing erroneous values in the data, which increases the accuracy of
the final result.
8.1
Camera and traffic parameters
In visual identification of vehicles, the ability to set different system parameters is an
advisable step. The configuration of the observed road section is static so it is possible to preset parameters describing the road, camera characteristics and normal
ranges of traffic parameters. Some systems rely on the relationship between the size
of the frame in pixels and the actual size of the observed road. It is even possible by
setting the actual coordinates of several points of the frame to determine threedimensional location and type of vehicles, without using stereoscopic camera [6].
Such configuration parameters can be:
 road length,
 number and lane boundaries,






driving direction for each lane,
real distance between virtual sensors,
mounting height of the video camera,
viewing angle or focal length of the lens,
central point of the intersection,
zones not subject to analysis.
8.2
Processing and filtering parameters
Noise filtering should be done in two stages. The first stage covers video frame processing before measuring vehicle parameters and the second involves the evaluation
of the estimated parameters.
The first stage could include reducing colour noise, contrast adjusting, colour normalization, application of smoothing filters, frame thresholding or difference between two consecutive frames and more. Reducing noise in the image is useful in
cases where optical flow is calculated. Normalization of frames is aimed at eliminating unevenness in brightness. The application of morphological operation "erosion"
on the binary image removes small objects occurring due to noise, rain, movement of
trees, birds, animals, etc. In order to eliminate false objects or those which are not
important, an area of interest must be set and only in this area objects are analyzed.
This is useful when is not possible to avoid areas close to the road with common
movements like sidewalks and parking spaces.
It is possible that even after frame filtering and separation of potential vehicles,
the extracted data contains objects that are not cars or with incorrect parameters due
to some reason. Such objects are moving with unreal speed, suddenly changing direction, got too large or too small. Characteristics of the vehicles slightly change
between successive frames and the big difference have to be considered as not normal and corresponding object filtered out. Example of a jump-like change of the
parameters of a vehicle is where two or more vehicles overlap visually or by their
shadows. Additional filtering parameters can be the valid ranges for vehicle size. For
example, motorcyclists and cyclists occupy significantly less space than cars, and
trucks and buses more space. This space can be measured during on-site trial tests
and later setup in the software. Only after this stage the system can proceed by reporting the real count of vehicles, calculation of additional traffic indicators and
checking for violations.
8.3
Vehicle detecting algorithms
In the theory of digital image processing, there are numerous operations. Different
analyzing algorithms apply different sets of operations for a given task and therefore
their efficiency and accuracy is varying and depends on many factors. The main task
of these algorithms is the detection of moving and stopped vehicles on the road.
Other task may include speed and vehicle type recognition. Development of fast and
reliable algorithm is a process requiring time and deep knowledge. Different weather
and road conditions may require the use of several methods to achieve accurate data
in a wide range of situations.
At present there are numerous methods for detecting objects in video footage.
They can be classified into 3 groups:
 methods using prior information about objects,
 methods based on movement or change in video footage,
 methods based on wave analysis.
Preliminary information used to identify an object as a vehicle can be symmetry,
colour, contours of the vehicle, the presence of round objects (wheels), 3D models
and headlights. Symmetry in the horizontal and vertical directions can be used to
distinguish vehicles from other objects. Due to the typical colour of asphalt and its
uniformity, opening the car can be based on colour differences. The presence of a
dark shadow is a sure sign of the presence of a car. This method is limited to places
with no side objects near the road.
On the basis of uniform colour and the lack of contours in the image and during
vehicle passage contours appear, which can be used to classify vehicle by type by
creating its wire 3D model. Change in a certain way of intensity of the image, also
could be signal for the presence of a vehicle.
Classification of vehicles according to the distance between the front and rear axle
(wheels) requires the camera to be positioned laterally and preferably perpendicular
to the road. Another method used during the night time is the detection of a pair of
punctuated bright objects representing vehicle’s headlights.
All methods discussed so far use spatial features to separate vehicles from the
background. Another used method is the calculation of optical flow. The vector field
of the moving object is called optical flow. Here several frames are used, i.e. the
temporal characteristics of the objects. The calculation of optical flow is a heavy
computational task that is only applicable to moving objects. This prevents the
method from applying on congested roads and intersections, because stopped vehicles can’t be detected.
Wave transformation is a new method for image processing. Movement of vehicles is described in full 3D spatial-temporal model covering several video frames.
8.4
Computation difficulty and CPU power
Another area influencing the final results and functionality of the system is the computational difficulty of the algorithms. Since it is important to obtain data in real
time, it is necessary to pay attention to the following factors:
 volume of the video signal,
 the efficiency of the processing algorithms,
 the performance of microprocessors.
Volume of the input signal W is defined as a product of width and height of the
frame in pixels multiplied by the number of frames per second and is measured in
pixels/s:
W = FW FH FS .
(1)
Where W - volume, FW - width of frames, FH - height of frames, FS - frames per
second. The bigger is the frame and higher the frame rate is, the larger the volume of
the input signal per second is, which requires a faster processor.
The implementation of additional filtration of the frames or more precise algorithms requires the calculations to be optimized. Common approach for speeding up
calculations is to use integer arithmetic, since this type of operations are faster compared to floating point operations. Another optimization can be reducing the volume
of the input signal via a pre-resizing of the frames by a given ratio smaller than 1.
Frame rate of 10 fps is sufficient to obtain accurate data, but typically the standard
TV speed of 25 or 30 fps is used to obtain smooth video.
9 Conclusion
The video sensors for traffic monitoring are well-suited for a wide range of transport
tasks because in addition to measuring individual vehicle parameters, they can also
detect traffic violations, queues of cars and unwanted situations such as traffic jams.
They provide coverage of a large number of lanes on a long road section, which
makes them attractive. The lack of necessity for stopping the traffic during installation and maintenance activities, as well as the possibility for using already installed
video cameras, makes this solution even more valuable for transport authorities. The
opportunity to expand their software functions remotely from the control centre,
without changing the cameras, makes from video traffic sensors a promising technology with long life.
The guidelines for future development in the sphere of video traffic monitoring
can be in the direction of improving the techniques and algorithms for vehicle detection in the cases of merging several nearby vehicles and working during night time.
There are video sensors using stereoscopic cameras addressing the problem with
merging of adjacent vehicles. These video sensors determine exact spatial location of
vehicles on the road and their type.
References
1. Mimbela, E., Klein, L.: A Summary of Vehicle Detection and Surveillance Technologies
used in Intelligent Transportation Systems, New Mexico State University (2007)
2. Lakov, V.: Application of video surveillance in traffic management, Utilities, ISSN: 13123017, Vol. 91 (2011)
3. Coifmana, B., Beymerb, D., McLauchlanb, P., Malikb, J.: [2] A real-time computer
vision system for vehicle tracking and traffic surveillance, PERGAMON Transportation
Research Part C 6 (1998)
4. Markishki, P.: The Magic of the Light, http://magic-light.photopia-bg.net/Photo1b.htm
(2010)
5. Bramberger, M., Pflugfelder, R. P., Maier, A., Rinner, B.: A Smart Camera for Traffic
Surveillance, Institute for Technical Informatics, Graz University of Technology
6. Han, D., Leotta, M., Cooper, D., Mundy, J.: Vehicle Class Recognition from Video based
on 3D Curve Probes, Brown University of Rhode Island
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement