Evaluation of Kolmogorov - Smirnov Test and

Telfor Journal, Vol. 7, No. 1, 2015.
Evaluation of Kolmogorov - Smirnov Test and
Energy Detector Techniques for Cooperative
Spectrum Sensing in Real Channel Conditions
Demian Lekomtcev, Student Member IEEE, and Roman Marsalek, Member IEEE
Abstract — The cognitive radio technology allows solving
one of the main issues of current wireless communication
technologies, namely a deficit of vacant spectrum. A dynamic
spectrum access used in the cognitive radio networks (CRN)
gives an ability to access an unused spectrum in real time.
Cooperative spectrum sensing is the most effective method
for spectrum holes detecting. It combines sensing information
of multiple cognitive radio users. In this paper, an
experimental evaluation of spectrum sensing methods based
on the Kolmogorov - Smirnov statistical test and Energy
Detector using the Universal Software Radio Peripheral
(USRP) devices synchronized through a MIMO cable and
with further processing in the GNU Radio and Matlab
software are presented. Three hard decision fusion schemes
are analyzed. Simulation comparison between these rules is
presented via Receiver Operating Characteristic (ROC)
curves. The influence of real channel with interferences is
compared in contrast to commonly assumed AWGN channel
model of vacant channel noise.
Keywords — Spectrum sensing, K-S test, Energy Detector,
Fusion Rules, Receiver Operating Characteristic (ROC),
ARLIER this year, the IEEE Standards Association
announced the creation of “IEEE forms study group to
explore standardization for spectrum occupancy sensing
technology” [1]. The main objective for this working
group is optimizing usage of radio spectrum for wireless
broadband services for the IEEE 802.22 standard. That is
why, despite the fact that there are many investigations
aimed at studying the spectrum sensing, this question is
Paper received April 2, 2015; revised June 15, 2015; accepted June
22, 2015. Date of publication July 15, 2015. The associate editor
coordinating the review of this manuscript and approving it for
publication was Prof. Branimir Reljin.
This paper is a revised and expanded version of the paper presented
at the 21th Telecommunications Forum TELFOR 2014.
This research has been supported by the MEYS of the Czech Republic
project LD12006, with the corresponding COST action IC1004. Thanks
also for partial support of the BUT Internal project FEKT-S-14-2177
(PEKOS). The measurements were performed in the laboratories of the
SIX center, registration number CZ.1.05/2.1.00/03.0072, the operational
program Research and Development for Innovation built with the help of
a project CZ.1.07/2.3.00/20.0007 WICOMT.
D. Lekomtcev is with the Department of Radio Electronics, Brno
University of Technology, Technicka 12, 61600 Brno, Czech Republic
(phone: 420-541146541; e-mail: xlekom00@stud.feec.vutbr.cz).
R. Marsalek is now with the Department of Radio Electronics, Brno
University of Technology, Technicka 12, 61600 Brno, Czech Republic
(phone: 420-541146582; e-mail: marsaler@feec.vutbr.cz).
still open.
The simplest and the most common method of spectrum
sensing is the Energy Detector (ED). Moreover the sensing
devices do not need any knowledge about the Primary
User (PU) signals. The Kolmogorov-Smirnov (K-S) test
[2] used in statistics to compare two random distributions
is one of the promising approaches to spectrum sensing. Its
application to this domain is based on the idea of testing
the measured signal distribution over the noise distribution
(usually AWGN). In this paper we want to perform an
extended analysis of K-S test and ED in more realistic
conditions – e.g. to test what is the influence of real vacant
channel noise with interferences and to evaluate the
influence of the precision of cumulative distribution
function estimation. Moreover we are also focusing on few
simple hard decision methods for cooperative sensing
because these technologies are effective, simple and lowcost to be implemented. The evaluation is based on the
experimental data from the Universal Software Radio
peripheral (USRP) devices controlled by GNU Radio
software with subsequent analysis in Matlab.
It is also important to note that most of the research
efforts on different fusion rules for cooperative sensing are
analytical and/or Matlab simulations based [3]-[5].
Recently a few measurement-based studies on cooperative
sensing can be found in [6]-[8]. In [2], [9] and [10] the
authors investigate an individual sensing based on K-S
test. These works provide the basis for our paper.
The rest of this paper is organized as follows. In
Sections II and III, we introduce K-S test-based and ED
based spectrum sensing algorithms, respectively. Section
IV briefly presents three most common hard fusion rules
for cooperative sensing. In Section V, we provide a
description of the experimental setup. Section VI presents
simulation results, while Section VII concludes the paper.
The detection of a signal within a noisy measure over a
specific frequency band is the key problem associated with
spectrum sensing. The essence of this problem is to
distinguish between the following two hypotheses:
n(t ),
r (t ) = 
s (t ) + n(t ),
Telfor Journal, Vol. 7, No. 1, 2015.
where r(t) is the received signal at Secondary User (SU)
location, s(t) represents the transmitted signal of the PU
observed at SU, and n(t) is a noise (in the simplest case the
AWGN, or other noise model).
In the case of K-S test, a decision between one or the
other hypothesis is performed as described below. This
test checks the accordance between the empirical
(measured) and the theoretical distribution functions,
whereas the distribution parameters of the theoretical
distribution are known in advance. Fig. 1 illustrates the
principle of this method.
that computes the chi-square CDF at each of the values in
xi using the corresponding degrees of freedom v (in our
case v=2).
The hypothesis H1 is accepted at the significance level α
(the false alarm probability) when the value of the test
statistic TKS is lower than a critical value k(α,N). There are
tables for the values of k(α,N) for 5 ≤ N ≤ 50 in the
literature. As in our investigation N ≥ 50 the critical values
can be approximated as [2]:
k(α , N ) =
ln  .
2⋅ N α 
The motivation for this paper was to investigate what is
the influence of noise having the distribution different
from the exact Gaussian (i.e. real measured noise), to
explore the influence of CDF approximation by finite (and
small) numbers of points (important for the efficient
implementation) and to check the potential of cooperation
between several (at least two) sensing nodes.
In the case of ED technique, spectrum sensing is
performed as follows. First Nsamples of the received signal
from the investigated channel are received by SU. Then
the signal is divided into Nsegments. Then the signal energy
for each segment is estimated e.g. as:
Fig. 1. Visual description of K-S test.
The null hypothesis H0 is assumed to be valid when the
empirical and theoretical probability distributions are not
statistically significantly different and the alternative
hypothesis H1 is assumed when these distribution are
significantly different. First, the empirical Cumulative
Distribution Function (CDF), i.e., CDF of the received
signal is estimated.
This function is defined by:
1 N
F ( x) =
 J ( x ( n) ≤ x )
N n =1
where J denotes the indicator function, which takes on a
value “1” if the input is true, and a value “0” otherwise, N
denotes the number of data samples from each given
signal segment. The largest absolute difference between
the empirical and the theoretical CDFs is used as the
goodness-of-fit statistic, given by [11]:
T KS = sup {F ( x ) − G ( x ) }
where G(x) is the known (expected) theoretical CDF. In
accordance with [11] this equation can be expressed as:
F ( xi ) − G ( xi )
T KS = max
where F(xi) denotes the value of CDF evaluated at the
point xi.
To calculate the theoretical CDF for the absolute value
of complex signal and additive white noise with real and
imaginary parts having normal distribution, the chi-square
distribution with two degrees of freedom is used [12]. In
Matlab this can be implemented by calling the function:
G ( xi ) = chi 2cdf ( xi , v ) ,
 (r (t ))
n =1
where Y represents the output of the energy detector which
serves as the test statistic. To make a decision about the
presence or absence of a signal, we introduce γ as the
threshold that varies depending on the noise variance σn2.
The probability of false alarm (Pfa) and detection
probability (Pd) for ED is given as follows: [13].
 γ −σ 2 
n 
P fa = P (Y > γ | H 0 ) = Q
σ 2 / N 
 n
 γ − σ 2 (1 − SNR ) 
 (9)
Pd = P (Y > γ | H1 ) = Q
 σ 2 / N ⋅ 2 ⋅ SNR + 1 
 n
where Q(x) is defined by the complementary distribution
function of the standard Gaussian and is given as
1 ∞ −u 2 / 2
Q( x) =
2π x
SNR is a signal to noise ratio, which can be defined in
terms of the signal and noise variance (σs2 and σn2) as
SNR = s .
σ n2
Therefore for a constant false alarm rate (Pfa) the
threshold is given by [13]
Q −1( P fa ) ⋅ δ n2
γ = δn +
Lekomtcev and Marsalek: Cooperative Spectrum Sensing in Real Channel Conditions
To improve signal detection, the cooperative spectrum
sensing can be used [14], [15]. In the prospective scenario,
there are M SUs that sense one PU. Each of the SUs makes
its own decision regarding the presence or absence of the
PU, and forwards the binary decision (1 or 0) to a fusion
center (FC) for further processing. SUs are located at a
negligible distance from each other, compared to the
distance from them to PU. Thus from a long-term
perspective the primary signal received by all the SUs has
the same local mean signal power. For simplicity, we have
first assumed that the noise, fading statistics and average
SNR are the same for each SU, and the channels between
SUs and FC are ideal (i.e. there is no loss of information).
A final decision on the presence of PU is made by k out of
M SUs and can be described by a binomial distribution
based on the Bernoulli trials where each trial represents
the decision process of each SU. The generalized formula
for the probability of detection at the fusion center is given
by [14], [15]:
M −l
M M 
P fa =   P fa,i 1 − P fa,i 
l=k  l 
M −l
M M 
Pd =   Pd ,i 1 − Pd ,i 
l=k  l 
where Pd,i and Pfa,i are the probabilities of detection and
false alarm, respectively, for each SU. In this paper three
rules of the hard combination scheme are discussed.
A. Logical OR-Rule
In this rule, a decision that the PU is present is made if
any of the SUs detects the PU. The cooperative probability
of detection (false alarm) using OR fusion rule can be
evaluated by setting k=1 in eq. (14, 15):
∏ (1 − P )
= 1 − ∏ (1 − P ) .
Pfa = 1 −
fa ,i
l =1
d ,i
l =1
B. Logical AND-Rule
In this rule, a decision that the PU is present is made if
all SUs have detected the PU. The cooperative probability
of detection (false alarm) using AND fusion rule can be
evaluated by setting k=M in eq. (14, 15):
Pfa =
l =1
Pfa ,i
P .
l =1 d ,i
C. Logical MAJORITY-Rule
In this rule, a decision that the PU is present is made if
half or more of the SUs detect the PU. The cooperative
probability of detection (false alarm) using MAJORITY
fusion rule can be evaluated by setting k=M/2 in eq. (14,
M −l
M M 
P fa =   P fa,i 1 − P fa,i 
l=M/2  l 
M −l
M M 
Pd =   Pd ,i 1 − Pd ,i 
l=M/2  l 
We use a similar measurement setup as the one used in
our previous paper [16]. Fig. 2 depicts this setup.
It consists of one personal computer (PC) and two
USRP2 devices. One of the USRP2 is carrying out SU1 as
well as PU roles and is connected to the PC through a
Gigabit Ethernet port.
Fig. 2. Experimental setup working in TV whitespaces.
The second USRP2 (which is carrying out a SU2 role) is
connected to the first USRP through a MIMO cable to
synchronize the system. Note that the precise
synchronization between the SUs must be realized in order
to realize the cooperative scheme. Optional GPSdisciplined oscillators provide the capability to
synchronize devices to the GPS standard over a large
geographic area [17].
In the setup, the first USRP implementing the SU1 and
PU employs the WBX daughter board (front-end), while
the SBX front-end is used in SU2 device. One SU had an
antenna behind the window and it was connected with an
amplifier to enhance the received signal, the other had it
on the laboratory table and the distance between them was
about 4 meters. With such a scenario, two received-signal
replicas were created.
Both USRPs work in the TV whitespace. The primary
user signal, which was transmitted by the first USRP, is an
8PSK signal with a bandwidth of 200 kHz (bandwidth
corresponding to wireless microphones bandwidth,
although the modulations are different) centered at 611
MHz carrier frequency, which corresponds to one of the
TV channels (in our experiment the 38th channel). A priori
we know that in the measurement location this channel is
vacant. Each SU scans two channels, the first one is a free
channel at 610 MHz frequency (this channel is vacant
throughout the experiment), the second one is a channel
where PU transmits. Note that although the channel is
vacant, it is possible to suppose that the character of
received signal is not exactly white Gaussian as it can
contain interferences from neighboring bands.
The PC is running Fedora 16, the signal processing
application is done using an open-source GNU Radio
version Each of SUs sensed the channel of interest.
By applying the GNU Radio the sensing results are
recorded into the data files for subsequent processing.
After that based on the recorded data files, the values of Pd
Telfor Journal, Vol. 7, No. 1, 2015.
and Pfa are calculated for each SU in Matlab in order to
estimate corresponding ROC curves. The equations to
estimate these quantities are as follows:
Number of segments that have Y > γ | H1
Pd =
Number of observed segments
Pfa =
Number of segments that have Y < γ | H1
Number of observed segments
In accordance with fusion rules for cooperative sensing,
a final decision about a vacant channel is made. The most
important results are presented below.
To evaluate the performance of a particular method of
(cooperative) spectrum sensing techniques, the key
characteristics as Pd and Pfa are used [3]. The ROC curve
graph visually shows the dependence between these two
parameters. All calculations for constructing diagrams are
performed in Matlab.
First, we analyzed the dependence of Pd and Pfa on the
SNR values. Fig. 3 demonstrates a higher detection
reliability for increased SNR values.
Fig. 4 is related to the practical implementation of K-S
testing device and describes ROC curves for different
numbers of points used for empirical and theoretical CDF
approximation (NCDF).
Fig. 4. ROC curves for 8PSK signal for different NCDF,
SNR=0.5 dB, 1000 segments with 100 data samples.
Note here that ROC curve evaluation has been done for
one hundred samples per segment only. This relatively
small value of the number of samples per segment (in
comparison with e.g. 1000-5000 samples in [2]) results in
a lower SNR performance.
Fig. 3. ROC curves for 8PSK signal for different SNR,
1000 segments with 100 data samples.
It can be observed that also the value of NCDF = 7
provides sufficient reliability only slightly degraded than
in the case of a higher number of points for approximation.
Note that in this case the number of points in each signal
segment and SNR value is fixed.
To illustrate the meaning of NCDF and its influence on
the empirical CDF, we present Fig. 5. It can be seen that
with increasing NCDF the empirical CDF becomes
smoother and closer to the theoretical one (in the case of
valid hypothesis H0).
Fig. 6 shows simulation results for different channels –
it compares the theoretical AWGN case with the real
measured channel in TV band. As expected and evident
from the figure, the pure AWGN channel results in a
higher detection probability Pd than the real fading
Fig. 5. Empirical (blue line) and theoretical (red line)
CDFs for different NCDF (top NCDF=7, in the middle
NCDF=50, bottom NCDF=100).
Finally, we compared the single spectrum sensing by
each SU with their cooperative version in accordance with
Lekomtcev and Marsalek: Cooperative Spectrum Sensing in Real Channel Conditions
three most-used fusion rules. Fig. 7 and Fig. 8 illustrate
these comparisons for K-S test and ED, respectively.
Fig. 6. ROC curves for 8PSK signal for different channel,
SNR=9 dB, 1000 segments with 100 data samples.
Fig. 7. ROC curves for KS test for 8PSK signal for three
fusion rules, SNR=0.5 dB, 1000 segments with 100 data
This is due to the fact that for OR rule the FC decides
whether a PU is present when at least one SU detects it.
While in the AND rule all SUs must detect a PU signal.
Also worth noting is the fact that K-S test has a better
detection performance than ED as for individual sensing as
well as for all cooperative fusion rules.
As it is known [18], for the IEEE 802.22 standard Pd
should be 90% or higher at Pfa = 0.1. In our experiment, all
fusion rules and each SU for K-S test meet this
requirement, whereas ED cannot reach it for the same
SNR values.
In this paper we present a performance evaluation of
Kolmogorov-Smirnov test-based and energy detectionbased spectrum sensing methods in close-to-real channel
conditions. We have compared the influence of real
channel noise and interferences on the sensing
performance expressed by the Receiver Operational
Characteristics and the effect of cumulative function
approximations by a finite number of samples. Finally, the
hard (binary) fusion rules for cooperative sensing were
used to demonstrate their improvement on the decision
process. Using an example of the sensed 8PSK signal with
different SNR values, it is first checked that the
performance of individual sensing improves with an
increased SNR. It was also demonstrated that the
probability of detection for real channel deteriorates in
comparison with AWGN usually assumed in theoretical
analysis. On the contrary, the effect of finite
approximation of distribution functions seems to be
In our future work, we will use the investigated sensors
for TV and wireless microphones signals cooperative
sensing in a city area on several vacant TV channels.
Fig. 8. ROC curves for ED for 8PSK signal for three
fusion rules, SNR=0.5 dB, 1000 segments with 100 data
As can be seen from Figs. 7 and 8, all cooperative
fusion rules gave a better detection performance than the
individual sensing. The AND rule has a better detection
performance than the OR and MAJORITY rules at low Pfa.
IEEE 802.22 Working Group on Wireless Regional Area Networks.
Enabling Rural Broadband Wireless Access Using Cognitive Radio
Technology in TV Whitespaces. Recipient of the IEEE SA
http://grouper.ieee.org/groups/802/22/ (May 31, 2014).
R. Marsalek, K. Povalac, “Kolmogorov – Smirnov test for spectrum
sensing: from the statistical test to energy detection,” IEEE
Workshop on Signal Processing Systems, pp. 97-102, 2012.
D. Teguig, B. Scheers and V. Le Nir, “Data fusion schemes for
cooperative spectrum sensing in cognitive radio networks,” Military
Communications and Information Systems Conference (MCC), pp.
1-7, 2012.
Md. Shamim Hossain, et. al., “Hard Combination Data Fusion for
Cooperative Spectrum Sensing in Cognitive Radio,” in International
Journal of of Elektrical and Computer Engineering (IJECE), vol. 2,
No.6, December 2012, pp. 818-818.
H. Du, et. al., “Hybrid Cooperative Spectrum Sensing Scheme
Using Double-Fusion in Cognitive Radio Networks,” in Journal of
Computational Information Systems, vol. 7, 2011, pp. 1500-1507.
S. Chaudhari, et al., “Measurement Campaign for Collaborative
Sensing using Cyclostationary Based Mobile Sensors,” IEEE
International Symposium on Dynamic Spectrum Access Networks
(DYSPAN), pp. 283-290, 2014.
S. Kyperountas, et al., “A Comparison of Fusion Rules for
Cooperative Spectrum Sensing in Fading Channels,” Virginia Tech
Symposium on Wireless Personal Communications, pp. 1-6, 2010.
A. Gameiro, et. al., “Selective reporting - a half signalling load
algorithm for distributed sensing,” in EURASIP Journal on
Wireless Communications and Networking, vol. 6, 2013, pp. 1-14.
Telfor Journal, Vol. 7, No. 1, 2015.
K. Povalac, R. Marsalek, “Application of the spectrum sensing
based on the Kolmogorov - Smirnov test to the OFDM resource
allocation,” in Recent Researches in Circuits, Systems,
Communications & Computers - Proc. of 2nd European Conference
of Communications (ECCOM'11), 2011, pp. 89-93.
H. Wang, “A Spectrum Sensing Algorithm Based on Kolmogorovsmirnov Two-sample Test,” in Journal of Computational
Information Systems, vol. 8, 2012, pp. 2601-2607.
W. Conover, Practical Nonparametric Statistics, 3rd Edition, John
Wiley and Sons, 1999, 592 p. ISBN: 978-0- 471-16068-7.
S. M. Kay, Intuitive Probability and Random Processes Using
MATLAB, New York, USA: Springer, 2006, ISBN 978-0-38724157-9.
P.R. Nair, A.P. Vinod and A.K. Krishna, “An adaptive threshold
based energy detector for spectrum sensing in cognitive radios at
low SNR,” IEEE International Conference on Communication
Systems (ICCS), pp. 574 - 578, 2010.
Yang, X. and Fei, H., Cognitive Radio Networks, 2nd Ed. Taylor &
Francis Group, LLC, 2009.
[15] D. Ruilong, et al., “Energy-Efficient Cooperative Spectrum Sensing
by Optimal Scheduling in Sensor-Aided Cognitive Radio
Networks,” IEEE Transactions on Vehicular Technology, vol. 61,
pp. 716 - 725, 2012.
[16] D. Lekomtcev, R. Marsalek, “USRP setup for energy detectionbased Cooperative Spectrum Sensing for Cognitive Radio
Networks,” IEEE Konference Zvůle 2014, pp. 41-44, 2014.
[17] Application Note Synchronization and MIMO Capability with
df (May 31, 2014).
[18] “IEEE Std 802.22™-2011 IEEE Standard for Information
Technology - Telecommunications and information exchange
between systems Wireless Regional Area Networks (WRAN) Specific requirements - Part 22: Cognitive Wireless RAN Medium
Access Control (MAC) and Physical Layer (PHY) Specifications:
Policies and Procedures for Operation in the TV Bands,” 2011.