Sparse Encoding of Signals through Structured Random Sampling by Praveen Kumar Yenduri A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Electrical Engineering: Systems) in The University of Michigan 2012 Doctoral Committee: Professor Anna C. Gilbert, Chair Professor Michael P. Flynn Associate Professor Clayton Scott Professor Jun Zhang c Praveen K. Yenduri All Rights Reserved 2012 To my beloved family, who constantly shower me with love, faith and support, though we are thousands of miles apart on opposite sides of the planet. ii ACKNOWLEDGEMENTS First of all, I would like to thank my advisor, Professor Anna Gilbert, for her constant support and guidance throughout the past few years. She has been a great mentor and source of inspiration. Without her, this thesis would not have been possible. I am also grateful to my committee members and co-advisors: Professor Michael Flynn, Professor Jun Zhang and Professor Clayton Scott. Their expert technical advice and expectations have constantly motivated me to achieve success. I would also like to thank my friend Jae Young Park for his support and encouragement. I appreciate his feedback on my proposal and thesis a lot. I would also like to express my gratitude to my friends and class-mates, Arun Padakandla, Madhu sudhan Reddy, Phani Motamarri, Supreet Jeloka, Kishan Kunduru, Jitendra Kochhar and many others who have made my stay pleasant and enjoyable. I am especially grateful to Janardhan and Jeenal Yandooru, my friends and family away from home, for their help and guidance in my time of need. They have been a constant source of kindness and have provided me the much needed love, fun and company during my lonely days in Michigan. I will always cherish the moments I had with many wonderful people I met in Ann Arbor. Peren Ozturan and Jillian Ong require special mention in this regard. It has been a true blessing to have a friend like Isha Patel, who has cheered and sup- iii ported me throughout my ups and downs. I am also thankful to Rajkumar, Sandhya, Sowmya; my friends back home in India, for their love and best wishes throughout the PhD journey. I am lucky to be blessed with a wonderful family, my parents, Murali and Prasanna, my sister Pavani and niece Akshara. I could not have come this far without the inspiration and support of my family, especially my mother. I will always be grateful for her dedication and the sacrifices she made for us. Last but not least, I would like to thank God for his continued blessings. Om Asato Maa Sadgamaya... Tamaso Maa Jyotir-Gamaya... Mrityor-Maa Amritam Gamaya... Om Shanti Shanti Shantihee !! Oh lord, lead us from unreality (of transitory existence) to the reality (of self). Lead us from the darkness (of ignorance) to the light (of spiritual knowledge). Lead us from the fear of death to the knowledge of immortality. Let there be peace everywhere !! iv TABLE OF CONTENTS DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi CHAPTER I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 1.2 1.3 Compressive sensing basics . Structured random sampling Contributions . . . . . . . . 1.3.1 Theoretical . . . . 1.3.2 Applied . . . . . . . . . . . . . . . . . . . . . 2 4 6 7 8 II. Random PPM (Pulse Position Modulation) ADC . . . . . . . . . . . . . . . 11 2.1 2.2 2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . Related Compressive Sampling (CS) . . . . . Hardware Design . . . . . . . . . . . . . . . 2.3.1 The PPM ADC Architecture . . . 2.3.2 The random PPM ADC Design . . 2.4 The random PPM ADC Implementation . . 2.4.1 Ramp Generator . . . . . . . . . . 2.4.2 Comparator . . . . . . . . . . . . . 2.4.3 Random clock and start generator 2.4.4 The Time-to-Digital Converter . . 2.5 The Reconstruction Problem . . . . . . . . . 2.6 The signal model . . . . . . . . . . . . . . . 2.6.1 The measurement matrix . . . . . 2.7 The Reconstruction Algorithm . . . . . . . . 2.7.1 Analysis of Algorithm . . . . . . . 2.8 Algorithm 2: Median of estimators (MOE) . 2.9 Experimental Results and Discussion . . . . 2.9.1 Simulation results . . . . . . . . . . 2.9.2 Prototype and measurement results 2.10 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 14 17 17 20 24 25 26 28 28 29 30 31 33 35 42 44 45 53 56 III. Model Of A Sparse Encoding Neuron . . . . . . . . . . . . . . . . . . . . . . . 58 v . . . . . . . . . . . . . . . . . . . . . . . . . 1 . . . . . . . . . . . . . . . . . . . . 3.1 3.2 3.3 . . . . . . . . . 58 61 62 63 64 66 68 71 73 IV. Continuous Fast Fourier Sampling . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.4 3.5 3.6 3.7 4.1 4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input stimulus model . . . . . . . . . . . . . . . . . . . . . . . . Time encoding with Integrate-And-Fire Neurons . . . . . . . . . 3.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Integrate-And-Fire Neurons with Random Thresholds The Low-Rate Integrate-and-Fire Neuron . . . . . . . . . . . . . The Reconstruction Algorithm . . . . . . . . . . . . . . . . . . . Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . 5.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.4 . . . . . . . . . . . . . . . . . . . V. Spectrum Sensing Cognitive Radio . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 5.2 5.3 . . . . . . . . . . . . . . . . . . . 75 77 77 78 80 80 82 83 85 89 4.4 4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . Background and preliminaries . . . . . . . . . . . . . . . . . 4.2.1 The problem setup and notation . . . . . . . . . . 4.2.2 The Ann Arbor Fast Fourier Transform (AAFFT) Continuous Fast Fourier Sampling . . . . . . . . . . . . . . . 4.3.1 Sample set construction . . . . . . . . . . . . . . . 4.3.2 The CFFS Algorithm . . . . . . . . . . . . . . . . . 4.3.3 Proof of Correctness of CFFS . . . . . . . . . . . . Results and Discussion . . . . . . . . . . . . . . . . . . . . . Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Wideband Spectrum Sensing Model . . . . . . . . . . . . . . . . . . . . 5.3.1 The structured random sampling system . . . . . . . . . . . . . . . 5.3.2 The Uniformly-Interleaved Filter bank (UIFB) . . . . . . . . . . . 5.3.3 Frequency Identification . . . . . . . . . . . . . . . . . . . . . . . . 5.3.4 Improving robustness through median operation . . . . . . . . . . . Simulation Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Varying Sub-sampling Ratio . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Varying Input SNR . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3 Varying R (Number of Frequencies per Channel) . . . . . . . . . . 5.4.4 Simple Heuristics for Estimating s (Number of Occupied Channels) Conclusion and Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi 91 93 95 95 96 100 103 106 107 107 109 110 115 116 LIST OF FIGURES Figure 1.1 Figure showing the on-grid and off-grid sampling. The crosses represent the Nyquist grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Block diagram of the PPM ADC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2 Waveforms depicting the sampling procedure in the PPM ADC . . . . . . . . . . . 17 2.3 Histogram of Correlation Coefficients between different pairs of columns of a signal dependent measurement matrix and a random measurement matrix (of size 15 x 40). The y-axis represents the number of correlation coefficients that fall in any particular bin of coefficient values. . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.4 Example probability distribution functions (pdfs) of τ , s and t = τ + s . . . . . . . 24 2.5 The random PPM ADC block diagram along with the TDC building blocks . . . . 25 2.6 Timing signals and comparison of operation between a regular PPM ADC and the random PPM ADC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.7 The ramp generator which is a component of the ADC in [1] . . . . . . . . . . . . . 27 2.8 Schematic of the comparator, a component of the ADC in [1] . . . . . . . . . . . . 27 2.9 Random start signal and random clock generation . . . . . . . . . . . . . . . . . . 29 2.10 Measurement matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.11 (a) Mean output SNR versus input SNR and (b) success percentage (fraction of trails that succeed) versus input SNR for 9-tone and 17-tone signals. The s-term NYQ (Nyquist) benchmark represents the best s-term approximation to the signal in frequency domain. Success means the correct identification of the frequencies of all tones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Reconstruction of a single tone signal with varying number of measurements (a) with no noise (b) success percentage when no noise (c) sampling needed for 99% success, with noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 2.13 Reconstruction of a 11 tone signal with varying amount of time jitter noise . . . . 49 2.14 Mean output SNR versus random PPM ADC sampling rate, for fixed bitrates of 4, 5 and 7 Mbps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.12 vii 2.15 Output SNR vs input SNR for a demodulated FM signal . . . . . . . . . . . . . . . 51 2.16 Output Vs Input SNR for a (a) multitone signal (b) demodulated AM signal with a sawtooth message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 2.17 Hardware setup for the random PPM ADC . . . . . . . . . . . . . . . . . . . . . . 54 2.18 Reconstruction of a single tone signal from samples collected by the regular and the random PPM ADC prototypes operating at varying sampling rates. The y-axis on the right displays the corresponding root mean square (rms) error. . . . . . . . 55 Reconstruction of a 5-tone signal from samples collected by random PPM with sampling rate at 8.65% of the Nyquist rate . . . . . . . . . . . . . . . . . . . . . . 56 3.1 Spike trains produced by an auditory neuron . . . . . . . . . . . . . . . . . . . . . 59 3.2 Time encoding with an integrate-and-fire (IAF) neuron . . . . . . . . . . . . . . . . 63 3.3 Sparse time encoding with Low-Rate integrate-and-fire(IAF) neuron. . . . . . . . . 66 3.4 Output SNR vs input SNR for signals with S = 10 . . . . . . . . . . . . . . . . . . 72 3.5 Output SNR vs input SNR for signals with S = 60 . . . . . . . . . . . . . . . . . . 73 4.1 Figure showing the samples acquired in AAFFT for each (t, σ) pair . . . . . . . . . 78 4.2 Figure showing the samples acquired by S1 (X’s) and the samples (O’s) required to apply AAFFT on B = [16, 47] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 4.3 Calculation of N -Wraparound t(1) from t . . . . . . . . . . . . . . . . . . . . . . . 81 4.4 Figure showing the arithmetic progression samples acquired in CFFS for a (t` , σ` ) pair and their wraparounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 The Sparsogram (time-frequency plot that displays the dominant frequencies) for a synthetic frequency-hopping signal consisting of two tones. The same sparsogram is obtained both by AAFFT (S1) and CFFS . . . . . . . . . . . . . . . . . . . . . . 85 4.6 Applying CFFS to different blocks of signal x . . . . . . . . . . . . . . . . . . . . . 86 4.7 Frequency-hopping signal with unknown block boundaries. . . . . . . . . . . . . . . 87 5.1 (left) The magnitude spectrum of a wideband signal (FN = 120MHz) with s = 5 occupied channels in a total of K = 64 channels. (right) The desired output of the spectrum detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.2 Block diagram of the spectrum sensing scheme . . . . . . . . . . . . . . . . . . . . 96 5.3 (top) Sampling pattern of the proposed structured random sampling scheme and (bottom) random samples of UIFB outputs . . . . . . . . . . . . . . . . . . . . . . 96 Ideal Pass-bands of filters F0 , F1 , .. in a (left) Regular sub-band decomposition filter-bank with R = 3 and (right) a uniformly-interleaved filter-bank (UIFB) with R = 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 2.19 4.5 5.4 viii 5.5 A conceptual block diagram of the uniformly-interleaved filter bank . . . . . . . . . 98 5.6 (left) Input signal spectrum with K = 4 channels (N = 25), (right) signal spectrum after uniform frequency interleaving through mapping f 7→ 19f mod 25 which corresponds to a time dilation t 7→ 4t mod 25 . . . . . . . . . . . . . . . . . . . . . 98 5.7 (top) Input signal spectrum with K = 4 channels (N = 25) and R = 6 frequencies per channel, (middle) signal spectrum after uniform frequency interleaving through mapping f 7→ 19f mod 25 which corresponds to a time dilation t 7→ 4t mod 25. Also shown are the R = 6 pass-bands of the sub-band decomposition filter bank, (bottom) signal spectrum at the output of the first filter in the UIFB. . . . . . . . 101 5.8 Figure showing the various terms in the linear system B(r)b(r) = y(r). 5.9 The spectrum detection scheme illustrated for a signal with s = 2 channels occupied in a total of K = 4, for R = 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.10 Pd (left) and Pf (right) vs. sub-sampling ratio for J = 1, 3, 5, 9 . . . . . . . . . . . 107 5.11 Pd (left) and Pf (right) vs. SNR for Nyquist-rate ED and for proposed scheme with J = 5, L/K = 0.35, 0.3, 0.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.12 (left) DTFT of a bandlimited signal with bandwidth 2W , (right) DFT of the same signal observed in a limited time window . . . . . . . . . . . . . . . . . . . . . . . . 109 5.13 Probability of detection Pd versus R and Spectral leakage (expressed as a fraction of total energy in an occupied channel) versus R . . . . . . . . . . . . . . . . . . . 110 5.14 Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 3, input SN R = −2 dB, s = 5 and different values of sin . . . . . . . . . . . . . . . . . 111 5.15 Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 5, input SN R = −2 dB, s = 5, sin = 8, with and without the estimation of s̃ using Heuristic A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 5.16 Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 5, input SN R = −2 dB, s = 5, sin = 8, with and without the estimation of s̃ using Heuristic B. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 ix . . . . . . 102 LIST OF TABLES Table 2.1 The Periodic Random Sampling Reconstruction (PRSreco) Algorithm . . . . . . . 35 2.2 Algorithm 2 : The Median of Estimators (MOE) . . . . . . . . . . . . . . . . . . . 43 2.3 Comparison of the PPMreco and MOE algorithms, used for signal reconstruction with random PPM ADC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.1 The Reconstruction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.1 The Continuous Fast Fourier Sampling (CFFS) algorithm . . . . . . . . . . . . . . 82 4.2 Percentage error in boundary identification . . . . . . . . . . . . . . . . . . . . . . 88 5.1 The Spectrum Sensing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 x ABSTRACT Sparse Encoding of Signals through Structured Random Sampling by Praveen Yenduri Chair: Anna Gilbert The novel paradigm of compressive sampling/sensing (CS), which aims to achieve simultaneous acquisition and compression of signals, has received significant research interest in recent years. CS has been widely applied in many areas and several novel algorithms have been developed over the past few years. However, practical implementation of CS systems remains somewhat limited. This is due to the limited scope of many algorithms in literature when it comes to the employed measurement architectures. In several CS techniques, a key problem is that physical constraints typically make it infeasible to actually implement many of the random projections described in the algorithms. Also, most methods focus only on discrete measurements of the signal, which is not always practicable. Therefore, innovative and practical sampling systems must be carefully designed to effectively exploit CS theory in practice. This work focuses on developing techniques that randomly sample in time, xi that are also characterized by the presence of some structure in the sampling pattern. The structure is leveraged to enable a feasible implementation of acquisition hardware, while the randomness ensures recovery of sparse signals via greedy pursuit algorithms. In certain cases, the presence of a predefined structure in the sampling pattern can be further exploited to obtain other advantages such as reducing the run-time of reconstruction algorithms. The main theme in the thesis is to develop algorithms that bridge the gap between theory and practice of structured random sampling. The work is motivated by several application problems where structured random sampling offers attractive solutions. One of the applications involves development of a low-power architecture for analog-to-digital conversion (ADC), that incorporates time-domain processing and random sampling techniques. Improving energy efficiency in both ways, the developed ADC occupies a unique position in the literature of compressive sensing ADCs. Similar techniques in structured random sampling are employed to develop a novel low-rate neuron model which encodes information present in sensory stimuli at a rate that is proportional to the actual amount of information present in the signal rather than its duration. The developed neuron model demonstrated superior performance in terms of sparse encoding and recovery error when compared to the neurons proposed earlier in the literature. Along with techniques borrowed from theoretical computer science, structured random sampling has been successfully employed in designing a novel, distributive, xii spectrum sensing scheme for application in wide-band cognitive radios. Simulations show that the proposed scheme exhibits a performance similar to that of a Nyquist rate method, even with high noise and severe under-sampling. Additional structure in random sampling was further utilized to develop a sophisticated, resource-efficient, continuous sampling and reconstruction algorithm for quickly approximating the frequency content of spectrally-sparse digital signals. xiii CHAPTER I Introduction Compressive Sampling/Sensing (CS) is a novel sampling paradigm that exploits the redundancy present in many practical signals and images of interest to recover them from far fewer samples or measurements, typically well below the number required by the Shannon/Nyquist sampling theorem [2]. CS achieves this through two key ideas : (1) Sparsely representing the signals of interest in an appropriate basis and (2) Employing random measurements (signal projections) to extract maximum amount of information using only a minimum number of measurements. A key problem with many CS techniques in the literature is that physical constraints typically make it infeasible to actually measure many of the random projections described in the algorithms. Therefore, innovative and sophisticated sampling systems must be carefully designed to effectively exploit CS theory in practice. In this work, we focus on techniques that sample in time (which can be treated as linear projections of Fourier coefficients). We develop random sampling algorithms, that are also characterized by the presence of some structure in the sampling pattern. The structure is leveraged to enable a feasible implementation of acquisition hardware, while the randomness ensures recovery of sparse signals via greedy pursuit algorithms. We are motivated by several application problems where structured random sampling offers 1 attractive solutions. Our theme is to develop algorithms that bridge the gap between theory and practice of structured random sampling. 1.1 Compressive sensing basics The basic idea of compressive sensing or compressive sampling is to exploit redundancy (i.e. sparsity or compressibility) in an input signal in order to reconstruct it from a small set of observations of the signal. In other words, compressive sensing aims for “smart” sampling of signals to acquire only the “important” information. In this way, the signal sampling rate can be reduced from the Nyquist rate to a rate that is proportional to the actual amount of information present in the input signal. The new sampling theory thus underlies procedures for sampling and compressing data simultaneously. Let the signal of interest be represented by a vector x of length N . We say that x is sparse if it contains only a few non-zero components compared to the total length (N ) of the signal. A compressible signal is one that is reasonably well approximated as a sparse signal. Let a vector y = Ax represent linear measurements taken from x by the measurement system. The matrix, A, is called the measurement matrix and has a size K x N , where the number of measurements K N . The reduction in the number of measurements that can be tolerated is proportional to the sparsity of the input signal x. The problem of recovering the signal x can be cast as that of solving an under-determined system of equations Ax = y. Solving for x based on y is an ill-posed problem in general, as there are infinitely many x that satisfy the relation Ax = y; however, it may be possible to uniquely solve for the input signal 2 x under the assumption that x is sparse or compressible. Of course, arbitrarily under-sampled linear measurements (i.e. arbitrary matrices A) will not succeed in recovering sparse vectors x. It has been shown that if the measurement matrix A satisfies the Restricted Isometric Property (RIP), then the sparse vector x can be recovered exactly [3]. A matrix is said to satisfy RIP with parameters (s, ) for ∈ (0, 1), if for all s-sparse1 vectors z, (1 − )||z||2 ≤ ||Az||2 ≤ (1 + )||z||2 Thus an RIP(2s, ) matrix A approximately preserves the Euclidean length of 2ssparse vectors, which in turn implies that A approximately preserves the distance between any two s-sparse vectors. For example, if x1 and x2 are two s-sparse vectors, then x1 6= x2 implies that A(x1 − x2) 6= 0. Thus the input x can be recovered by searching for the sparsest vector z that satisfies the condition, Az = y or ||Az −y||2 ≤ in case of noisy measurements. x can be expressed as the solution to the following optimization problem: arg min||z||1 such that ||Az − y||2 ≤ There is no known algorithm that can verify if a given matrix is RIP other than the exponential time brute force algorithm. However various results have been published about the RIP nature of the matrix A if it is drawn from certain distributions of random matrices. For example, in the cases where A is a random Gaussian matrix [4], a random Bernoulli matrix [5] or a random partial DFT matrix [4], A satisfies RIP(s, ) with high probability, if the number of measurements K > O(−2 slogO(1) N ) (O(.) refers to the Big-O notation [3]). Algorithms that carry out the `1 −minimization through linear programming to find x are typically referred to as the Basis Pursuit 1 An s-sparse vector has at most s non-zero elements. 3 (BP) algorithms. BP algorithms are usually significantly slower (in theory) when compared to greedy pursuit algorithms ([6],[7],[8]), which limit the search space for x to s-sparse vectors, for a given s. Greedy pursuit algorithms try to minimize the `2 -norm of the error (defined as Ax − y) subject to the condition that x is s-sparse: min||Ax − y||2 such that ||x||0 ≤ s Conventional greedy pursuit algorithms such as those proposed in [7] and [8], require the matrix A to be RIP. The assumption of sparsity of x is justified by the fact that real world signals are often sparse or compressible in some transform domain. For example, communication signals such as FSK (frequency shift keying) are sparse in Fourier domain and natural images are often sparse in a Wavelet domain. In other words, even if the input signal x is not sparse, it can be represented as x = W X for some sparse vector X, where W denotes the sparsifying transform matrix. The net measurement matrix now changes to B = AW for the system BX = y. In this work, we are interested in input signals which are sparse in the frequency (Fourier) domain. In that case, X is the DFT of x and W is the IDFT (inverse discrete Fourier transform) matrix. 1.2 Structured random sampling Most CS algorithms use RIP matrices whose entries are obtained independently from a standard probability distribution (e.g. random Gaussian measurements). However, such matrices are highly impractical and not feasible for real-world applications. In fact, very often the physics of the sensing modality and the capabilities of sensing devices limit the types of CS matrices that can be implemented in a specific 4 application. In contrast, in many applications, sampling in time can be efficiently implemented through the use of analog-to-digital converters. More over, sampling in time is the best option for frequency sparse signals, since time domain and frequency domain are maximally incoherent [9]. Hence, we restrict our attention to matrices that correspond to sampling in time. Now, it is also possible to construct deterministic sampling schemes that result in measurement matrices that satisfy the RIP property [10]. However, they require far more measurements when compared to random sampling schemes (O(K 2 ) vs O(K)). Hence, we focus on random sampling. Applications, however, often do not allow the use of completely random matrices, but put certain physical constraints on the measurement process. This leads us to structured random sampling. The structure can be used to achieve a feasible implementation. In certain cases, the structure can also be used to obtain faster recovery algorithms. However imposing structure onto random sampling also has its disadvantages. The resultant measurement matrices do not necessarily satisfy the RIP condition, leading to the need to develop new algorithms and analysis. Depending on the structure imposed, there might also be other undesired consequences which have to be dealt with. Figure 1.1: Figure showing the on-grid and off-grid sampling. The crosses represent the Nyquist grid. 5 Random sampling can be broadly classified into two categories, shown in Fig. 1.1. In on-grid random sampling the time points at which the signal is sampled are chosen randomly from a Nyquist grid (represented as crosses in the Fig. 1.1). On-grid random sampling can be viewed as random sub-sampling of a digital signal. Where as, in off-grid random sampling the time points have a continuous distribution and do not have to lie on the Nyquist-grid or any other grid. If the signal is observed during a time interval I, the off-grid time points at which the signal is sampled are continuous random variables with their range in I. The distribution of the random variables depends on the application. An example of an off-grid random sampling device is the random PPM ADC (presented in Chapter II). Another example is a level-crossing ADC, which samples the signal when it crosses some predefined amplitude levels. Several fast sub-linear time algorithms can recover s-sparse signals from random on-grid samples ([11],[12],[13],[14]). These algorithms have a storage requirement and runtime of O(s logO(1) N ) (with the exception of [14], which samples the signal at an average rate close to Nyquist rate). A specific case of random off-grid sampling is studied in [15]. In this work, we develop both on-grid and off-grid structured random sampling techniques and investigate their application to different problems of interest. 1.3 Contributions This thesis treats both theoretical and application aspects of structured random sampling. The main contributions of this thesis work, in structured random sampling, 6 are the following ([13],[16],[17],[18],[19],[20],[21]). 1.3.1 Theoretical • Periodic random sampling reconstruction (PRSreco) algorithm (Section 2.7): We developed a new algorithm [16][20] for recovering frequency-sparse signals, from off-grid time samples, obtained in a periodically random pattern, at a sub-Nyquist rate. In periodic random sampling, the input signal is sampled at a random point within an interval of certain length and this process is repeated in every subsequent interval of that length. The algorithm is used for reconstruction in randomized time-based analog to digital converters that implement periodic random sampling. We analyze the algorithm and provide bounds on the reconstruction error. The PRSreco falls under the general category of greedy pursuit algorithms, but does not require the measurement matrix to be RIP. We also take a non-conventional approach in proving the error guarantees. Reference [16]: P.K. Yenduri, A.C. Gilbert, M.P. Flynn, and S. Naraghi, “Rand PPM: A low power compressive sampling analog to digital converter,” IEEE International Conf. on Acoustics, Speech and Sig. Processing (ICASSP), pp. 5980 − 5983, May 2011. • Continuous fast Fourier sampling (CFFS) algorithm [13] (Chapter IV): Fourier sampling algorithms use a small number of structured random samples to quickly approximate the DFT of a spectrally-sparse digital signal from a given time window or block. Unfortunately, to obtain the spectral information on a particular block-of-interest, the samples acquired must be appropriately 7 structured for that block. Thus the sampling pattern forces a block-wise analysis and does not accommodate an arbitrary block analysis. We developed a new sampling procedure called Continuous Fast Fourier Sampling (CFFS) which samples the signal at sub-Nyquist rates and permits a sub-linear-time analysis of arbitrarily blocks of the signal. Thus, CFFS is a highly resource-efficient continuous sampling and reconstruction algorithm. Reference [13]: P.K. Yenduri and A.C Gilbert, “Conitnuous fast fourier sampling,” In Proceedings of Sampling Theory and Applications (SAMPTA), Marseille, France, 2009. 1.3.2 Applied • Random PPM: A low power compressive sampling time based ADC [16] [20]: A random pulse-position-modulation (PPM) ADC architecture is proposed in Chapter II. A prototype 9-bit random PPM ADC incorporating a pseudo-random sampling scheme is implemented as proof of concept. This approach leverages the energy efficiency of time-based processing. The use of sampling techniques that exploit signal compressibility leads to further improvements in efficiency. The random PPM (pulse-position-modulation) ADC employs compressive sampling techniques to efficiently sample at sub-Nyquist rates. The sub-sampled signal is recovered using the PRSreco algorithm, which is tailored for practical hardware implementation. We develop a theoretical analysis of the hardware architecture and the reconstruction algorithm. Measurements of a prototype random PPM ADC and simulation, demonstrate this theory. The prototype successfully demonstrates a 90% reduction in sampling rate compared to the Nyquist rate for input signals that are 3% sparse in frequency domain. 8 Reference [20]: P.K. Yenduri, A. Rocca, A.S. Rao, S. Naraghi, A.C Gilbert, and M.P. Flynn, “A low power compressive sampling time-based analog to digital converter,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), Special Issue on Circuits, Systems and Algorithms for Compressive Sensing, Oct. 2012. • LowRate IAF: A sparse encoding model of neuron [17] [18] (Chapter III): Neurons as Time Encoding Machines (TEMs) have been proposed to capture the information present in sensory stimuli and to encode it into spike trains. These neurons, however, produce spikes at firing rates above Nyquist rate, which is usually much higher than the amount of information actually present in stimuli. We propose a low-rate neuron which exploits the sparsity or compressibility present in natural signals to produce spikes at a firing rate proportional to the amount of information present in the signal rather than its duration, while using the spiking information in a smart manner to improve the performance of stimulus recovery. Reference [17]: P.K. Yenduri, A.C. Gilbert, and J. Zhang, “Model of a sparse encoding neuron,” Twenty First Annual Computational Neuroscience Meeting (CNS), Jul. 2012. Reference [18]: P.K. Yenduri, A.C. Gilbert, and J. Zhang, “Integrate-andfire neuron modeled as a low-rate sparse time-encoding device,” Proceedings of Third International Conference on Intelligent Control and Information Processing (ICICIP), Jul. 2012. 9 • Compressive and collaborative spectrum sensing for wide-band cognitive radios [19] (Chapter V): One of the primary tasks of a cognitive radio (CR) is to monitor a wide spectrum and detect vacant channels, which can then be used for secondary transmission opportunities (i.e. for transmission by unlicensed users). CR systems thus enable dynamic spectrum access (DSA) and improve the overall efficiency of spectrum usage. However, the requirement of prohibitively high sampling rates to monitor a wideband, makes this a challenging task. In this work, we present a novel wideband spectrum sensing model that reduces the sampling requirement to a sub-Nyquist rate, proportional to the number of occupied channels in the wide spectrum. The sampling scheme is efficiently implementable using low-rate analog-to-digital converters (ADCs). The sensing algorithm uses techniques borrowed from theoretical computer science and compressive sampling, to detect the occupied channels with a high probability of success. The algorithm is implementable for spectrum sensing in a single CR, as well as in a decentralized CR-network with minimal communication between one-hop neighbors. The algorithm also has many other attractive features which make it different from other algorithms in literature. Reference [19]: P.K. Yenduri and A.C. Gilbert, “Compressive, collaborative spectrum sensing for wideband cognitive radios,” The Ninth International Symposium on Wireless Communication Systems (ISWCS), Aug. 2012. 10 CHAPTER II Random PPM (Pulse Position Modulation) ADC 2.1 Introduction Applications of low-power ADCs include power constrained wireless environmental sensing, high energy physics and biomedical applications such as massive-parallel access of neuron activity ([22],[23],[24]). We present a new low-power, compressivesampling analog to digital converter which we call a random PPM ADC. The random PPM ADC is one of the first ADCs that takes advantage of the combination of time-based analog-to-digital conversion techniques and compressive sampling. In addition, we discuss a new reconstruction algorithm called PRSreco (Periodic Random Sampling reconstruction) and present theoretical upper-bounds for input signal reconstruction error. This algorithm is tailored to make it viable for practical hardware implementation. Technology scaling generally improves power consumption and speed, however, it poses a number of challenges in the design of ADCs. Scaling reduces the supply voltage, which in turn reduces the signal dynamic range. This has the direct effect of reducing signal to noise ratio (SNR). One way to overcome the problems of lowvoltage design is to process signals in time domain. Technology scaling favors time 11 domain processing since it reduces gate delays and thus improves time resolution. A wide variety of time-based ADCs that quantize time or frequency instead of voltage or current, have been proposed. These designs include simple architectures such as single-slope analog to digital conversion [25], pulse width modulation (PWM) ADC [26], asynchronous level crossing designs [27], VCO-based Σ∆ modulators [28] and integrate and fire circuits ([29],[30]). Continuous time DSPs are proposed in [31]. A continuous time level crossing ADC, such as [32] and [33] can be attractive for slow moving signals. However, the the key advantages of these devices are lost if continuous time DSP is not available. Furthermore, sparse signals can be dominated by high frequency content. This work expands on the pulse position modulation ADC architecture developed in [1]. The PPM ADC is itself an elaboration of the PWM architecture in which a continuous-time comparator compares the input to a periodic ramp, to convert the input signal information to a time-domain representation (see Section 2.3). A two-step time-to-digital converter (TDC) then converts the time domain information to digital domain. With the use of a two-step TDC, the PPM ADC achieves both high resolution and high dynamic range, along with low power consumption. Another way to obtain an improvement in the power efficiency of an ADC is to reduce the sampling rate ([34]) since to a first order, power consumption is proportional to sampling frequency. We can achieve this by employing random sampling techniques that exploit the redundancy (i.e. compressibility or sparsity) of the input signal to reduce the sampling rates to below the Nyquist rate. We implement random sampling by introducing randomness into the reference ramp signal used by the PPM ADC. The proposed random PPM ADC lies at the intersection of time-based ADCs 12 and compressive-sampling ADCs, and thus improves efficiency in both ways. Many compressive sampling (CS) ADC architectures and acquisition systems have been proposed in recent years. While some designs lack efficient implementation of CS encoding or decoding (reconstruction) algorithms in hardware ([35],[36]), other designs focus on efficient compression but not the optimization of power consumption ([37],[38]). Some compressive sensing designs, such as [39], employ a conventional high-speed ADC as an integral component. Also, none of the above designs use time-based conversion techniques to reduce power consumption. The random PPM ADC, thus occupies a unique position in the literature of compressive sensing ADCs. The remainder of the chapter is organized as follows. Section 2.2 briefly relates the random sampling techniques used in this work with compressive sampling techniques that reduce the number of measurements needed to store and reconstruct a given input signal. The PPM and random PPM architectures are discussed in Section 2.3. A prototype random ADC, implemented as a custom CMOS PPM ADC coupled to an FPGA (Field-Programmable Gate Array) is described. The hardware implementation of the random PPM ADC is described in Section 2.4. The problem of reconstructing the input signal from ADC output samples is introduced in Section 2.5. In Section 2.7, we develop the PRSreco algorithm for the recovery of input signals that satisfy the signal model presented in Section 2.6. This new algorithm falls under the general category of greedy pursuit methods that aim to minimize the norm of the reconstruction error, subject to the sparsity conditions of the input signal. The PRSreco is analyzed in Section 2.7.1. The error bound of the recovered signal is discussed in Section 2.7.1. A second reconstruction algorithm called the 13 MOE (median of estimators) is developed and analyzed in Sec. 2.8. The appendices contain details about the mathematical modeling of the randomized sampling system along with lemmas and theorems that provide proof of correctness and run-time details of the algorithms. The PRSreco algorithm and the MOE algorithm can also be used for signal reconstruction in other randomized time based ADCs as the analysis in Section 2.7.1 is easily extended. The algorithms are tailored to reduce computational cost and thus are viable for practical hardware implementation. Our analysis along with the numerical simulations and experimental results presented in Section 2.9, show that a random sampling time-based ADC exhibits much better performance than a nonrandom ADC operating at sub-Nyquist sampling rates. 2.2 Related Compressive Sampling (CS) Consider the under-determined system of equations BX = y, where B is the net measurement matrix of size K x N , X is the DFT of the input signal vector x of length N and y is the measurement vector of length K (K < N ). The sparse spectrum X can be expressed as the solution to the following optimization problem1 [40] : arg min||X||0 such that BX = y The above problem requires the solution of a non-convex combinatorial problem, which is not practical [41]. Hence the `0 -“norm” in the objective function is often 1 arg min||X|| solves for X that has the smallest ` -“norm”, where ||X|| is defined as the number of non-zero 0 0 0 elements in X. 14 replaced by its convex relaxation, the `1 -norm2 . That is, arg min||X||1 such that BX = y It has been shown that if the measurement matrix B satisfies the Restricted Isometric Property (RIP), then the sparse vector X can be recovered exactly [3]. Algorithms that carry out the `1 −minimization through linear programming to find X are typically referred to as the Basis Pursuit (BP) algorithms. Many of the proposed CS ADCs ([35],[36],[39]) use BP algorithms for reconstruction. However, BP algorithms are challenging to implement in hardware and are usually significantly slower when compared to greedy pursuit algorithms ([6],[7],[8]). Greedy pursuit algorithms try to minimize the `2 -norm of the error (defined as BX − y) subject to the condition that X is s-sparse: min||BX − y||2 such that ||X||0 ≤ s Conventional greedy pursuit algorithms such as those proposed in [7] and [8], require the matrix B to be RIP. The measurement matrix B associated with the PPM ADC, does not necessarily satisfy the RIP condition. If a new matrix B is constructed randomly for each input signal X, then the RIP condition on B can be relaxed ([11],[6]). Hence, we impose the condition that B be a random matrix (newly constructed for each input signal X) and develop a new reconstruction algorithm, that falls under the category of greedy pursuit algorithms, but does not require matrix B to be RIP. Different CS algorithms offer different error guarantees. We call the error guarantee `2 /`1 , if the following is true: C ||X − X̃||2 ≤ √ ||X − Xs ||1 s 2 The `1 -norm of a vector is defined as the sum of the absolute values of its elements. 15 where X̃ is the output of algorithm, C is a constant and Xs is the best s-term representation of X. A stronger error guarantee is the `2 /`2 , given by: ||X − X̃||2 ≤ C||X − Xs ||2 CS algorithms also offer different kinds of failure guarantees. Some methods fix the measurement matrix B and prove the reconstruction results for all input sparse signals X, while other algorithms can prove the reconstruction results with high probability for each input sparse signal X and a random measurement matrix B. Algorithms such as the BP [3], [7], [8] offer the stronger “for all” guarantee, but the weaker `2 /`1 error guarantee. On the other hand Fourier sampling algorithms ([11],[12],[13]) offer stronger `2 /`2 error guarantees and weaker “for each” failure guarantee. If a new measurement matrix B is randomly chosen for each sparse signal X, then a “for each” failure guarantee is sufficient. The output of an ADC is usually evaluated in terms of the reconstruction SNR, which involves the ratio of `2 norm of the signal to the `2 norm of the reconstruction error. Thus, a `2 /`2 error guarantee is more suitable for such an analysis. The only way to obtain a `2 /`2 error guarantee is to have a “for each” failure guarantee. In this work, a “measurement” of the input signal is a measurement of the amplitude of the input signal at some time point. To obtain a random measurement matrix B, the input signal is sampled at random time points. The measurement vector y represents the amplitude of the signal at those random time points. Sub-linear time algorithms ([11],[12],[13],[14]) can recover s-sparse signals from random on-grid samples. However, PPM ADC produces off-grid samples. A specific case of random off-grid sampling is studied in [15] with a number of measurements, K > O(sR2 log(4N/)), where R is the dynamic range of X and is a tolerance parameter. In this chap16 ter, we deal with reconstruction from signal-dependent3 , random, off-grid samples. Thus the problem setting is different from [15], leading to an algorithm that offers different error guarantees and different conditions for recovery. We also take a nonconventional approach in proving the error guarantees. Unlike the random on-grid sampling techniques, our algorithm does not achieve a sub-linear run-time. 2.3 Hardware Design In this section we describe the PPM ADC architecture and the design of the random PPM ADC design. Figure 2.1: Block diagram of the PPM ADC Figure 2.2: Waveforms depicting the sampling procedure in the PPM ADC 2.3.1 The PPM ADC Architecture A block diagram of the PPM ADC is shown in Fig. 2.1. The sampling proce- dure [1] is depicted in Fig. 2.2. A comparator continuously compares the input signal 3 The time points at which the signal is sampled depend on the signal, in contrast to being completely deterministic or completely random. 17 with a reference voltage ramp. An output pulse is generated by the comparator at the time instants where the ramp voltage exceeds the input signal. The time elapsed between the beginning of the ramp and the instant the input signal crosses the ramp (i.e. s1 , s2 , .. as seen in Fig. 2.2) is measured and quantized by a two-step 9-bit time to digital converter (TDC). The simplest form of a TDC is a digital counter, however, to achieve a high resolution, one needs to have a very high counter frequency which in turn leads to a large energy consumption. On the other hand, delay line circuits [42] are more energy efficient for time measurement, however, the delay line must be long to measure long periods of time and can suffer from non-linearity. As a compromise, the two-step TDC consists of a 5-bit counter which performs coarse quantization and a delay line TDC as the fine quantizer that resolves 4-bits. By combining a low frequency counter and a delay line TDC, the two-step TDC thus achieves both energy efficiency and a large dynamic range. Detailed implementation of the TDC is discussed in [1]. The output of the ADC is a sequence of time duration measurements si , which represent the relative position of the output pulse in every ramp period. Since the signal information is encoded into the position of the pulse, the ADC is called pulse position modulation ADC. The starting points of the ramps are given by τi = (i − 1)T , where T is the period of the reference ramp signal. The crossover times are ti = τi + si . If we assume the slope of the ramp is a constant m, the signal amplitude at the crossover times is yi = msi . In this way, from the output {si } of the PPM ADC, we can calculate the sample set {(ti , yi ), i = 1, 2, ..}. Note that T is also the average sampling period of the ADC, because the ADC takes one sample within every interval of T seconds. 18 Non-uniform signal dependent sampling: If we make the approximation that yi are samples at uniform time points τi instead of the non-uniform ti , we see harmonic distortion in the frequency spectrum of the recovered signal. Linear low-pass filtering is a straightforward conventional technique for constructing uniform samples from non-uniformly sampled information. According to [43] an oversampling factor of at least 8 is needed to use the traditional low pass filtering technique. Another approach is to use a time-varying iterative non-linear reconstruction method, (as described in [1]) which allows the signal to be sampled closer to the Nyquist rate. Let us represent measurement vector y as y = Sx where S is the nonuniform sampling operator. Let operator P represent a low pass filter with cut-off frequency tuned to Nyquist frequency. The algorithm described in [1] is as follows: x0 = y = Sx x1 = P x0 = P Sx xi+1 = P (y − Sxi ) + xi , for i = 1, 2, .. It is easy to see that xi = P S( i P (I − P S)k )x, where I is the identity opera- k=1 i P tor and (I − P S)0 = I. Limi→∞ (I − P S)k = (P S)−1 and thus Limi→∞ xi = k=1 −1 (P S)(P S) x = x. When the PPM ADC is operated at Nyquist rate, applying low pass filter to non-uniform samples causes harmonic distortion which can be corrected through the iterations. However below Nyquist rate, applying a low pass filter to non-uniform samples causes severe aliasing in the frequency domain which cannot be rectified through iterations. In other words, The operator P S cannot be inverted through the algorithm used. Thus, the method still requires the sampling frequency 19 to be above the Nyquist rate (the oversampling factor of 8 is brought down to 2). Further, a sufficient condition of si < T /4 is required to obtain a stable sampling set [44]. If this condition is relaxed, there is no guarantee that the algorithm converges. For sampling rates below the Nyquist rate, the method diverges. Our goal is to convert the PPM ADC into a compressive sampling ADC so that the signal can be recovered from samples acquired at sub-Nyquist rate. The sampling system and the reconstruction algorithm are co-designed to achieve this. A Regular PPM ADC at sub-Nyquist sampling rate: A straight-forward way to operate a PPM ADC as a compressive sampling ADC is to increase its average sampling period T (which is also the reference ramp period). The sampling frequency F can be brought down to a value F < FN , where FN is the Nyquist frequency of the input signal. We refer to this sampling architecture as the regular PPM ADC. We use the algorithm proposed in Section 2.7 for reconstruction. However, since the time points ti (calculated in Section 2.3.1) are non-random and highly signal dependent, the resultant measurement matrix B is also non-random and thus disobeys the design rules of random sampling algorithms. In order to fit into the compressive sensing framework and to meet the criteria for successful signal reconstruction, we need to make some modifications to the PPM ADC sampling system. A random sampling scheme is introduced in the next section. 2.3.2 The random PPM ADC Design Simple theoretical results from sparse approximation state that a low corre- lation between different columns of a measurement matrix indicates the possibility of better signal recovery [6]. Albeit crude, this is one of the elementary methods for 20 Figure 2.3: Histogram of Correlation Coefficients between different pairs of columns of a signal dependent measurement matrix and a random measurement matrix (of size 15 x 40). The y-axis represents the number of correlation coefficients that fall in any particular bin of coefficient values. evaluating a measurement matrix with respect to its reconstruction properties. Consider the following simple experiment to motivate the introduction of randomness into the PPM ADC. Let B c be the measurement matrix that relates the DFT of the input signal to the samples obtained by the PPM ADC at sub-Nyquist rate with an average sampling period of T . As discussed, the time points ti at which a signal is sampled by the PPM ADC are signal dependent. Now let B r be the corresponding measurement matrix, when in each interval [(i − 1)T, iT ] of length T , the signal is sampled at a time point ti that is uniformly distributed on [(i − 1)T, iT ]4 . Ideally, we want any measurement matrix B to be orthonormal (B H B = I) so that the input signal can be easily recovered as B H y = B H BX = X. An orthonormal matrix is characterized by a zero correlation between any two columns of the matrix. Fig. 2.3 plots the histograms of correlation coefficients between different columns, for both the signal dependent B c and the random B r matrices. As can be seen from the figure, in the case of a signal dependent B c several columns have high correlation 4 Note that in the actual implementation of the random PPM ADC, we don’t have complete control over t = τ +s , i i i and so we randomize τi instead. 21 coefficients. On the other hand for the random matrix B r , coefficients are all distributed in the left region of the plot. Intuitively, since B r achieves closer to zero correlation coefficients when compared to B c , it is more “orthonormal” than B c and is expected to lead to a better signal recovery. Motivated by this observation, we introduce randomness into the PPM ADC system. We convert the ramp starting times τi (which are deterministic in regular PPM ADC) into random variables. More specifically, let τi − (i − 1)T ∼ Uniform[0, T ], ∀i. That is, in each interval [(i − 1)T, iT ] of length T , the reference ramp has a random starting point. We call this architecture random PPM, as the ramp starting times are now randomly and independently chosen. As before, the crossover times are ti = τi + si and the signal amplitude at the crossover times is yi = msi , where m is the slope of the ramp. Assume that the duration of the reference ramp, that is the time for which the ramp is greater than zero in each interval [(i − 1)T, iT ], is given by cT for some 0 < c < 1. For the original PPM, c ≤ 0.25, so as to satisfy the stability condition for the reconstruction algorithm presented in [44]. Choosing τi − (i − 1)T ∼ Uniform[0, T ], ∀i can cause overlap between adjacent ramps. For example, when τ1 = T and τ2 < T + cT there is an overlap. There are two ways to deal with this issue. The first is to adjust the distribution of τi as τi −(i−1)T ∼ Uniform[0, T −cT ], ∀i. The implemented prototype random PPM ADC uses this adjustment. Another way is to employ a second sampling system. The sampling systems each produce a ramp in alternate periods and sample the input alternatively in each period. Thus there will be no overlap in the ramps and the original choice of distribution for τi can 22 be maintained, that is τi − (i − 1)T ∼ Uniform[0, T ], ∀i. Note that τi are all still independently chosen. Also note that the net power consumption can be kept almost same as before since the two samplers would sample at half the rate as before. Even though we do not actually use two sampling systems in the prototype random PPM ADC, we assume that the overlap between adjacent ramps is allowed for theoretical simplicity. The presented “mathematical framework” thus closely matches the implementation (actual or thought experiment). The time points at which the random PPM ADC samples the signal are given by ti = τi + si . To further aid the analysis, we assume that the phase φ of the input signal x(t + φ), is uniformly distributed in [0, 2π]. This induces a probability distribution on si . The probability density function (pdf) of ti can be obtained by convolving the pdfs of τi and si (since τi and si are independent for all i). Dropping the i for convenience, let h(r), 0 ≤ r ≤ cT be the pdf of s (not to be confused with sparsity of the signal s). Recall that cT is the on-time of the reference ramp in each period. The pdf of τ is pτ (r) = 1/T, 0 ≤ r ≤ T . Thus convolving the two it is easy to see that pdf of t = τ + s is given by H(q) , 0 ≤ q ≤ cT T pt (q) = 1 , cT ≤ q ≤ T T ) 1−H(q−T , T ≤ q ≤ T + cT T where H(q) is the cumulative distributive function of the random variable s. An example of h(r) and pt (r) is shown in the Fig. 2.4 below. These results are used in the proof of Lemma II.1 in Sec. 2.5. 23 Figure 2.4: Example probability distribution functions (pdfs) of τ , s and t = τ + s 2.4 The random PPM ADC Implementation Fig. 2.5 shows a block diagram of the random PPM ADC. The individual blocks are explained in detail in the following sections. A random clock generator block produces two outputs, a start signal (which goes high at each τi ) and a random clock. The reference ramp is generated only when the start signal is high. The comparator compares the output of the ramp generator with the input signal and generates a stop signal when the ramp voltage exceeds the input signal. The random clock acts as the time reference in the time-to-digital conversion (TDC) block. The two-step TDC (with a 5-bit coarse quantizer and a 4-bit fine quantizer) measures the time elapsed between the rising edges of the start and the stop signal. A synchronizer ensures correct alignment of the coarse and the fine time measurements. The ramp generator, comparator and the two-step TDC are implemented in 90 nm digital CMOS, while the random clock generator is implemented on an FPGA. Some of the key timing signals, shown in Fig. 2.6, provide a comparison of operation between a regular PPM ADC and the random PPM ADC. The regular PPM ADC receives a regular periodic clock with period Tclk . The start signal of a regular PPM ADC goes high at the beginning of each repetition period T . On the other 24 Figure 2.5: The random PPM ADC block diagram along with the TDC building blocks hand, the random PPM receives a random clock, which consists of regular clock cycles only when the start signal is high. The start signal for the random PPM ADC goes high after a random time tRAN D in each interval (of length T ). The start signal remains high only for a time tH and then goes low for the rest of the interval. The time tH is related to the slope m of the ramp such that the ramp covers the entire voltage range of the input signal in a time tH . During the time [tRAN D , tRAN D + tH ] when the start is high, the ramp is generated and when it crosses the input signal, the random PPM ADC makes one measurement (denoted as s1 in the figure). This process repeats in every interval [(i − 1)T, iT ], i = 1, 2, ... Therefore, the average sampling frequency of the ADC is F = 1/T . 2.4.1 Ramp Generator The ramp generator circuit is shown in Fig. 2.7. Charging a capacitor with a constant current produces the ramp signal. Cascoded PMOS transistors M3 and M4 implement the current source while M1 and M2 are the digital switches that control 25 Figure 2.6: Timing signals and comparison of operation between a regular PPM ADC and the random PPM ADC capacitor charging. The switches are, in turn, controlled by the start signal. The capacitor discharge is achieved simply with a switch to ground [1]. 2.4.2 Comparator The comparator is continuous time and is made up of two stages. The circuit is shown in Fig. 2.8. The first stage is a differential to single amplifier with a PMOS input pair. This is followed by an NMOS common source stage. The PMOS input pair operates in the subthreshold region. This is done to minimize power consump- 26 Figure 2.7: The ramp generator which is a component of the ADC in [1] tion and to provide a larger input common mode range, which allows for a larger dynamic range in the ramp [1]. Figure 2.8: Schematic of the comparator, a component of the ADC in [1] 27 2.4.3 Random clock and start generator The random waiting times tRAN D ’s (in Fig. 2.6) are produced using a linear feedback shift register (LFSR) system as shown in Fig. 2.9. Although the output of this system is only pseudo-random, with a large bit length sufficient randomness is achieved. The LFSR bit string is initialized with a non-zero seed. This bit string gives the number of regular clock cycles that the start signal initially remains low, i.e., tRAN D = (LFSR)Tclk , where LFSR stands for the (integer) value of the bit string. The start signal then goes high and stays high for a time tH (which is chosen such that mtH is greater than the input signal voltage range. tH is also chosen to be a multiple of Tclk ). To complete the interval of length T , the start signal is kept low for an additional T − tH − tRAN D seconds as shown in Fig. 2.6. Once a complete interval of length T has elapsed, the LFSR sequence is advanced to its next state and the same process is repeated with the new value of LFSR (thus, a new tRAN D = (LFSR)Tclk ). The random clock is produced by gating the start signal with the regular clock as shown in Fig. 2.9. The rising edges of start and the random clock are thus synchronized. Note that two short bit length LFSR systems can be coupled to produce a pseudo-random sequence with sufficiently large period. 2.4.4 The Time-to-Digital Converter The two-step TDC [1] measures the time interval (s1 in the Fig. 2.6) between the rising edges of the start signal (which is synchronous with the random clock), and the stop signal generated by the comparator. To enable correct alignment of the coarse and fine time measurements, the synchronizer block generates two additional signals, clk stop and counter enable. The clk stop signal is set by the arrival of the second 28 Figure 2.9: Random start signal and random clock generation rising edge of the clock after the stop signal. The counter enable signal is set by start and reset by clk stop. A 5-bit counter (which is the coarse quantizer) measures tc (in Fig. 2.6), which is the number of clock cycles elapsed while the counter enable signal is high. The slope of the ramp is designed such that tc is always less than 32 clock cycles. The fine TDC measures the time tf between the stop signal and clk stop signal rising edges. The overall TDC output is s1 = tc − tf . The fine TDC consists of a 32-element delay line, spanning two full clock cycles (the fine TDC thus divides one clock cycle into 16 equal slices and resolves 4 LSBs). 2.5 The Reconstruction Problem We now formulate the problem of reconstructing the input signal from samples collected by an ADC (regular PPM or random PPM). The samples are assumed to be collected at a sub-Nyquist rate. Let an N -length vector X represent the input signal in the Fourier domain. Let K (K < N ) be the number of measurements taken by the ADC. Let (ti , yi ), i = 1, .., K denote the measurements obtained from 29 the output of ADC (see Section 2.3). The time ti is the ith time point at which the ADC samples the input signal and yi is the signal amplitude at that time. Note that K N = F FN < 1, where F = 1/T is the average sampling frequency of the ADC and FN is the Nyquist rate of the input signal. We relate the input signal X with the measurement vector y through the equation BX = y, where B is the measurement matrix. The goal is to solve for X from BX = y. Note that X is the N -point DFT of the time domain input signal x. The reconstruction is done in the frequency domain, as the input signal is assumed to be sparse in frequency domain as indicated in the following input signal model. 2.6 The signal model In this paper we focus only on a subset of band limited signals that are band limited to [−W, W ]. The Nyquist rate of the input signal space is FN = 2W . If the input signal is sampled at Nyquist rate for a time of tT otal , then the number of samples N = FN tT otal . We assume that the input signal is s-sparse or s-compressible in the frequency domain. A signal is called s-sparse in the frequency domain, if the DFT of the signal samples at Nyquist rate has only s non-zero terms. A signal is called s-compressible5 in frequency domain, if the sorted list of its DFT coefficients has only s significant or dominant terms, compared to which the other terms are negligible. The input signal can be expressed as a linear combination of complex exponentials as follows: x(t) u s X cm exp(j2πfm t) m=1 where fm , m = 1, .., s are the s dominant frequencies which lie in the interval [−W, W ] and cm are the corresponding coefficients. We further assume that the input signal is 5 We call X, s-compressible, if it is well approximated as a s-sparse signal, ||X − X −α for some (s) ||2 ≤ C.s constants C and α > 0, where X(s) is the s-sparse signal that best approximates X. 30 real, hence s is even and one set of frequencies are the negative of the other set. Some practical signals that are frequency-sparse include frequency-hopping communication signals, narrowband transmissions with an unknown carrier frequency that can lie anywhere in a wide band, communication to submarines, radar [45] and geophysical [46] signals such as slowly varying chirps, etc. 2.6.1 The measurement matrix Figure 2.10: Measurement matrix To determine whether a successful signal recovery is possible from BX = y, we analyze the properties of the measurement matrix B, which is shown in Fig. 2.10. The matrix B can be intuitively constructed by making the observation that if f is the only frequency with a non-zero coefficient in the DFT X, i.e., in time domain x(t) = exp(j2πf t), then the samples of x(t) at time points ti are given by {exp(j2πf ti ), i = 1, .., K}. Putting f = (n/N )FN (as in a N -point IDFT), the samples form the nth column of the measurement matrix, for n = [−N/2 : N/2 − 1] (if N even) or [(−N − 1)/2 : (N − 1)/2] (if N odd). Hence, for a given i and n, 31 Bi,n = exp(j2π Nn FN ti ). It is to be noted that B is not a sub-matrix of the N -point IDFT matrix, since ti are non-uniform and do not lie on any Nyquist grid. We now look at the correlations between different columns of the random PPM H Bn denote the correlation between the nth measurement matrix B. Let Cnm = Bm and mth columns of B. The Lemma II.1 provides a tight upper-bound, on the order of 1/N , for the magnitude of expected correlation between different columns of B. A small expected correlation implies a better signal recovery, as discussed in Section 2.3.2 and illustrated in Fig. 2.3. Lemma II.1. Let If = [−N/2 : N/2 − 1] (if N even) or [−(N − 1)/2 : (N − 1)/2] (if N odd). For n, m ∈ If and n 6= m, |E(Cnm )| ≤ O (2.1) 1 N Proof. E(Cnm ) = H E(Bm Bn ) K X 2π 1 =E exp(j (n − m)ti ) K N TN i=1 ! K X 1 E (exp(θti )) = K i=1 where θ = j2π(n−m) N TN for convenience and TN = 1/FN . Now, using the distribution of t1 derived in Sec. 2.3.2, it can be proven that, E (exp(θt1 )) ≤ (exp(θT ) − 1) exp(θcT ) θT Now it is easy to obtain that E (exp(θti )) = exp(θ(i − 1)T )E (exp(θt1 )) 32 Hence, 1 E(Cnm ) ≤ K K X ! exp(θ(i − 1)T ) E (exp(θt1 )) i=1 1 exp(θKT ) − 1 E (exp(θt1 )) K exp(θT ) − 1 exp(θKT ) − 1 = exp(θcT ) θKT = Now (using | exp(θcT )| = 1 and then N TN ≤ KT < (N + 1)TN ) we have, exp(θKT ) − 1 |E(Cnm )| ≤ θKT KT = sinc (n − m) N TN ≤ sinc((n − m)(1 + 1/N )) ≤ 1/(N + 1) ≤ 1/N 2.7 The Reconstruction Algorithm The random PPM ADC samples the signal at a rate proportional to its finite rate of innovation, defined as the number of degrees of freedom per unit time [47]. For the signal model considered in this paper, the rate of innovation is given by s, the number of frequencies present in the signal. Algorithms have been proposed in [47] that can recover the s frequencies and their coefficients by using only 2s consecutive uniform samples from the signal. However, these algorithms cannot be applied 33 with the random PPM ADC as they require the samples to be uniformly spaced at Nyquist rate. Also, the measurement matrix B associated with random PPM ADC, is a signal-dependent non-uniform random Fourier matrix, and as such, does not necessarily satisfy the Restricted Isometry Property (RIP) assumed in [7] or the conditions assumed in [8]. This leads to the need to develop different algorithms with different theoretical analysis. A probabilistic approach is presented in Section 2.7.1. We call the developed reconstruction algorithm, Periodic Random Sampling reconstruction (PRSreco). A pseudo-code for the PRSreco algorithm is presented in Table 2.1. From Lemma II.1, we see that correlations between different columns of B are small on average. Hence, B H y is a good approximation to the signal X. In particular, the largest components in B H y provide a good indication of the largest components in X. The algorithm applies this idea iteratively to reconstruct an approximation to the signal X. At each iteration, the current approximation induces a residual, which is the part of the signal that has not been approximated yet. The current approximation vector X̃ is initialized to a zero vector and the residual is initialized to the measurement vector y. For a vector z, supp(z) is defined as set of indices of the non-zero elements of z and z(s) stands for the best s-term approximation6 of z. For an index set T ⊂ {1, 2, .., N }, zT stands for a sub-vector of z containing only those elements of z that are indexed by T . Similarly BT stands for a sub-matrix of B containing only the columns of B indexed by T . The algorithm initially obtains an estimate for the dominant frequencies in the signal through least squares and then refines the estimate of the set of dominant frequencies and their coefficients in an iterative fashion. 6 The best s-term approximation of a vector z can be obtained by equating all the elements of z to zero, except the elements that have the top s magnitudes. 34 PRSreco algorithm input: N (signal length), s (sparsity), (ti , yi ),i = 1, 2, .., K. output: X̃ (s-sparse approximation to X, length N ) (0) X̃ = 0, residual H r = y T =supp [B y](2s) −1 H X̃T = BTH BT BT y (0) (0) r = r − BT X̃T (Least Squares) for i = 0, 1, 2, .. X̃ (i+1) = X̃ + B H r(i) X̃ = [X̃ (i+1) ](s) r(i+1) = y − B X̃ until ||r(i+1) ||2 does not vary within a tolerance θ. Table 2.1: The Periodic Random Sampling Reconstruction (PRSreco) Algorithm The computationally intensive step of least squares is performed only once in the PRSreco algorithm. The least squares is implemented using the accelerated Richardson iteration [48] with runtime of O(sKlog(2/et )) where et is a tolerance parameter. The structure of the measurement matrix lends us to use the inverse NUFFT [49] with cardinal B-spline interpolation for forming the products of the form B H r, in a runtime of O(N logN ). Hence the total runtime of the algorithm is dominated by O(IN logN ) where I is the number of iterations. 2.7.1 Analysis of Algorithm Lemma II.2 says that the estimators of coefficients of X in the PRSreco algorithm produce close to correct values and their second moments (variances) are bounded. The results of Lemma II.2 and Lemma II.1 are used to prove Theorem II.3. 35 Lemma II.2. If number of measurements K = O(s/2 ) then for any s-sparse (or sH compressible) vector X, each estimate of the form X̃m = Bm BX for m = 1, 2, .., N , satisfies E(X̃m ) = Xm ± O (2.2) Var(X̃m ) ≤ (2.3) 1 N ||X||1 2 ||X||22 s Proof. For any vector X, E(X˜m ) = E N X ! H Bm Bi Xi = E Xm + i=1 = Xm + X N X ! Cim Xi i=1,i6=m E(Cim )Xi i6=m The required result is now true from Lemma II.1. 2 Now we will compute Var (X˜m ) = E(X˜m ) − (E(X˜m ))2 . But first, consider the following: H Cim C`m X K K 2π(`−m)tq n 1 X j 2π(m−i)t j NT N TN N = 2 e e K n=1 q=1 K 1 X j N2π (m−i+`−m)tn TN e = 2 K n=1 2π(`−m)tq n 1 X X j 2π(m−i)t j NT N TN N + 2 e e K n q6=n After applying expectation, and using that tn and tq are independent for n 6= q and also using the Lemma II.1,we see that the second term above can be ignored as it is O( N12 ). 36 2 H Hence E(Cim C`m ) u E(C`i )/K. We will use this in the expansion for E(X˜m ) as follows, N X H 2 E(X˜m X˜m ) = Xm + H E(C`m )Xm X` `=1,6=m N X + E(Cmi )XiH Xm + i=1,6=m X X E(C`i ) i6=m `6=m N X 2 + ≤ Xm K XiH X` H E(C`m )Xm X` `=1,`6=m + N X E(Cmi )XiH Xm i=1,i6=m + N X 1 2 X K i i=1,i6=m since, by Lemma II.1,E(C`m ) ≤ 1/N, is negligible for ` 6= m. We can similarly obN N P P 2 H tain the expansion (E(X˜m ))2 = Xm + E(C`m )Xm X` + E(Cmi )XiH Xm + `=1,`6=m N P N P i=1,i6=m E(Cmi )E(C`m )XiH X` . The last term can be ignored (assuming signal i=1,i6=m `=1,`6=m sparsity s N ). Now, H Var(X˜m ) = E(X˜m X˜m ) − (E(X˜m ))2 ≤ N X 1 2 2 Xi ≤ ||X||22 K s i=1,i6=m Once the PRSreco algorithm gets an approximation X̃ of X, it subtracts the contribution of the current approximation from the measurements and proceeds to recover the leftover signal X − X̃. As we move on to higher iterations of the algorithm, the energy in the leftover signal goes down, bringing down the upper-bound on the variance of the estimators (from Lemma II.2 applied to X − X̃). Thus a better approximation is obtained for the signal X in each higher iteration until the 37 required tolerance is reached or the algorithm converges. Please refer to the proof of Theorem II.3 for further details. Theorem II.3 offers an error guarantee for a signal recovered using the PRSreco algorithm and establishes the conditions on the sub-sampling ratio K/N that can be achieved using the random PPM ADC. If X is s-sparse and there is no noise in the measurements obtained from the random PPM ADC (operating at a sub-sampling ratio of K/N ), then from Theorem II.3, signal X can be recovered exactly. If the measurements are corrupted by some noise (e.g. quantization noise), the `2 -norm of the reconstruction error is bounded above by the `2 -norm of the noise. Theorem II.3. Let y = BX + ξ be the time domain samples of signal X obtained by the random PPM ADC, where ξ is an arbitrary noise contamination in the measurements and B is the resultant measurement matrix of size KxN . Let the phase7 φ of the time domain input signal x(t + φ) be uniformly distributed in [0, 2π]. Suppose |X[s] |2 ≥ 2α||X||22 /s + |X[s+1] | for some constant α and a given sparsity parameter s, where |X[i] | is the magnitude of the ith largest element of X. Given the error tolerance in reconstruction θ and K = O(s logN/2 ), with probability > 1−O(2 ) the algorithm produces an s-term estimate X̃ of signal with the following property, (2.4) ||X − X̃||22 2 c(B)||ξ||22 2 ||X − X(s) ||2 ≤ max θ , + 1−α 1−α where X(s) is the best s-term approximation of X. The runtime of the algorithm is O(IN logN ) where I = Number of iterations, with a gross upper bound of I < max(logN,log(||X||2 /θ)). The net storage requirement is O(N )+O(sK). The constant c(B) depends on the measurement matrix B. 7 That is, the time t = 0 at which we start to observe the signal, is assumed to be random. This induces a probability distribution on the signal dependent si . 38 Proof. First we will show that the PRSreco algorithm succeeds in identifying the top s terms of the signal. We will then derive the error guarantee. Let us begin with signal X exactly s-sparse. For simplicity lets assume that Xi , i = 1, .., s are the non-zeros. There exists a β < 1 such that |Xi |2 ≥ β||X||2 /s. (1) Let 0 < 2α ≤ β. For i = 1, .., s, using the Chebyshev inequality we have, Pr(|X̃i − Xi |2 ) ≥ α||X||2 ) s 2 (1) ≤ Var(X̃i )/ α||X|| ≤ s 2 ||X||2 α||X||2 / s s = 2 α (using Equation (2.3) from Lemma (II.2)). Hence Pr (1) X̃i good α||X||2 2 (1) 2 = Pr |X̃i − Xi | ≤ ≥1− s α Let |Xmin | be the smallest non-zero in X. Again using Chebyshev inequality, for each (1) of Xi , i > s we have Pr(|X̃i |2 ≤ |Xmin | − 1− 2 β−α = 1− 2 ( α α||X||2 ) s 2 ≥ Pr(|X̃i |2 ≤ (β − α) ||X|| )≥ s (1) since 2α ≤ β). Now, define Bernoulli random variables zi as indicators of failure of the ith coefficient estimator. That is Pr(zi = 0) = 1 − Ps i=1 zi for all i = 1, .., N . Let Z1 = 2 = 1 − Pr(zi = 1) α and Z2 = PN i=s+1 zi . We have s E(Z1 ) s2 /α 42 Pr Z1 > ≤ ≤ = . 4 s/4 s/4 α (1) (1) Hence Pr(No. of good estimators among X̃1 ,X̃2 ,...,X̃s (1) ≥ 3s ) 4 ≥ 1− 42 . α Note that the factor 1/4 is chosen as an example to simplify the presentation of the proof. Now lets move on to the 2nd iteration of the algorithm. More than 3s/4 estimators which were good in the first iteration are still good in the second iteration. This (2) is because the estimator X̃i depends on the same random correlations (between (1) Bi and other columns of B) as the estimator X̃i from the first iteration. Put the current approximation X̃ = [X̃ (1) ](s) as defined in the PRSreco algorithm (see 39 Table 2.1). Now for those coefficients whose estimators were not good in the first iteration we have, (2) |X̃i Pr α||X − X̃||2 − Xi | ≥ s 2 ! ≤ 2 2 ||X − X̃||2 /s = α α||X − X̃||2 /s like before, using the Equation (2.3) from Lemma (II.2) applied to X − X̃. Now (2) define a new Z1 (2) for these estimators. Note that E(Z1 ) ≤ s2 4α (since there are less (2) than s/4 terms in the definition of Z1 ). Now as before we have s 42 (2) Pr Z1 > 2 ≤ . 4 α Hence by the end of second iteration number of good estimators among the i = 1, .., s is ≥ 3s 4 + 3s 44 with a net probability ≥ (1 − 42 2 ). α Going on this way at k th iteration, number of good estimators ≥ (1 − ( 14 )k )s, with probability ≥ (1 − 42 k ) . α Similar statements can be obtained about Z2 , i.e., about the estimators with i > s. Hence after sufficient number of iterations (say I), all the estimators are good which implies that all the non-zero terms will be identified by the algorithm with Pr(Success) ≥ (1 − 42 2I 8I2 ) ≈ (1 − ) = 1 − O(2 ) α α after absorbing some constants along with number of iterations I into the number of measurements. If I is the sufficient number of iterations at which all estimators are good, then (1/4)I N < 1 ⇒ I = 0.5 logN = o(logN ). Hence an increase in number of measurements by a factor of log N is required. Note that the above is a gross lower bound for the success probability. In reality since all the estimators are highly dependent, the probability that they will be good together is higher than the product of the individual success probabilities, which is the gross lower bound produced by the above theory. 40 Now let the signal X be s-compressible (hence not exactly s-sparse). We start with K =O(s/2 ) as before. Again for simplicity let the first s elements of X be the h top s terms. For i > s we assume that |Xi |2 ≤ γ||X||2 /s for some γ < 1. Let Xmin t be the smallest coefficient in the head (i = 1, .., s) of X. Similarly let Xmax be the largest coefficient in the tail (i > s) of X. All the above arguments hold again except that for i > s the probabilities will involve γ in the following manner. For example in the first iteration, 2 t h − |X | Pr |X̃i − Xi |2 ≤ |Xmin | − (α)||X|| max s 2 ≥ Pr |X̃i − Xi |2 ≤ ( β1 − α − γ) ||X|| s ≥1− β2 1−(α+γ)β =1− 2 α (assuming 0 < 2α ≤ β − γ). Repeating the arguments from above we show that the algorithm succeeds in identifying the top s-terms. Now let us prove the error guarantee. Let us assume that ξ = 0 for the moment. Lets say the algorithm correctly identifies the position of top s terms in I iterations. (k+1) For any k > I,at iteration k + 1, |(X̃i − Xi )|2 < α||X − X̃ (k) ||2 /s for i = 1, .., s. Summing up the s inequalities we get, ||X̃ (k+1) − X(s) ||2 ≤ α||X − X̃ (k) ||2 where X(s) is the best s-term approximation to X. Now, ||X − X̃ (k+1) ||2 ≤ ||X − X(s) ||2 + ||X(s) − X̃ (k+1) ||2 ≤ ||X − X(s) ||2 + α||X − X̃ (k) ||2 . This implies, ||X − X̃ (k+1) ||2 ≤ 1−αk−I ||X 1−α − X(s) ||2 + αk−I ||X − X̃ (k−I) ||2 . For k large enough we have, ||X − X̃||2 ≤ 1 ||X − X(s) ||2 1−α 41 This is consistent with Equation 2.4. Now let ξ = Bn for some vector n. we have y = B(X + n). Following the arguments as before, we have, ||X + n − X̃||2 ≤ ||X+n−(X+n)(s) ||2 1−α ≤ ||X+n−X(s) ||2 1−α (since (X + n)(s) is the best s-term approximation to X + n). Now, ||X − X̃||2 ≤ ||X + n − X̃||2 + ||n||2 ≤ ||X+n−X(s) ||2 1−α + ||n||2 ≤ ||X−X(s) ||2 1−α + (2−α)||n||2 . 1−α Equation 2.4 by putting ||n||2 ≤ c||ξ||2 . This is true for some c(B) < We will have 1 σmin (B) where the denominator is the smallest singular value of B. 2.8 Algorithm 2: Median of estimators (MOE) Note that in the PRSreco algorithm (Sec. 2.7), the input signal was sampled for a total time of t = N/FN to get K samples. Instead if we sample the signal for a duration of mt, we get m copies of K measurements,with each set of measurements from a block of time t. Assume that the set of top s frequencies in the signal remains the same in all the m blocks of time (their coefficients can change). Then we can take a median over the estimators from different blocks to improve the identification of the top s frequencies. The idea of taking a median instead of mean was used in the count sketch algorithm [50] which estimates the most frequent items in a data stream. We propose to use the algorithm in Table 2.2 to identify and estimate the top s frequencies in the signal. Let B(i) be the measurement matrix formed (as shown in Section 2.6.1) from the time points in the ith block of time, for i = 1, .., m. Similarly let y(i) be the vector of measurements obtained from the ith block. 42 MOE algorithm input: N (Block length), m (No. of Blocks), s (sparsity) (t` , y` ),` = 1, 2, .., mK. output: X(i) for i = 1, .., m (signal in each block) Identification: For j = 1, .., N , H X̂j = median |B(1)H j y(1)|, .., |B(m)j y(m)| T = supp([X̂]s ) Estimation: For i = 1, .., m, −1 [X̃(i)]T = B(i)H B(i)H T B(i)T T y(i) Table 2.2: Algorithm 2 : The Median of Estimators (MOE) Theorem II.4. For m = O(ln( Nδ )), the MOE algorithm correctly identifies the set T of top s frequencies in the signal with Pr(Success) ≥ 1 − δ. Proof. Note that the arguments in proof of Theorem II.3 hold for all the m blocks of time. Let X̃ij = B(i)H j y(i) for i = 1, .., m and j = 1, .., N . Note that X̂j =median(|X̃ij |, i = 2 1, .., m) for j = 1, .., N . From Theorem II.3’s proof, Pr(|X̃ij | good) ≥ 1 − α = p(say), for i = 1, .., m. Assuming p > 0.5, from Chernoff bound we have Pr(X̂j good) 2 ≥ 1 − e−2m(p−0.5) ≥ 1 − δ 0 for m =O(ln( δ10 )). Hence Pr(Success) = Pr(X̂j good for j = 1, .., N ) ≥ (1 − δ 0 )N ≈ 1 − δ for δ 0 = δ/N . Note that the computationally intensive step of least squares is performed only once (per block of the signal) in both the algorithms. The least squares was implemented using the accelerated Richardson iteration [48] with runtime of O(sKlog(2/et )) where et is a tolerance parameter. The structure of the measurement matrix lends us to use the inverse NUFFT [49] with cardinal B-spline interpolation for forming the products of the form B H r, in a runtime of O(N logN ). Hence the total runtime of PRSreco algorithm is dominated by O(IN logN ) where I is the number of iterations. The per block runtime of the MOE algorithm which has only one iteration 43 is O(sKlog(2/et ))+O(N logN ), which is much less than that of the PRSreco algorithm. However as mentioned in section 2.8 to apply the MOE algorithm the signal has to satisfy the required additional condition of maintaining the same dominant set of frequencies throughout the observed time. Also the MOE algorithm processes the signal in blocks of m unlike the PRSreco algorithm. The sampling percentage (K/N = mK/mN ) is the same for both the algorithms. (100% sampling implies sampling at Nyquist rate.) These statements are summarized in Table 2.3. PRSreco Signal model Output Total number of operations % Sampling Operation efficiency Table 2.3: 2.9 MOE Same set of dominant s frequencies in all m blocks (coefficients Any sparse signal can vary) Estimate of signal of Estimate of signal of length N on length N each of m blocks O(IN logN ) per block O(mN logN ) per m blocks K/N mK/mN = K/N O(IN logN ) N O(mN logN ) mN Comparison of the PPMreco and MOE algorithms, used for signal reconstruction with random PPM ADC. Experimental Results and Discussion The regular PPM and the random PPM sampling architectures (described in Section 2.3) are implemented in hardware. The ADCs combined with the reconstruction algorithms are also simulated in MATLAB. A series of experiments compares the performance of the algorithms for both the sampling architectures. The Signal-to-Noise Ratio8 (SNR), which is defined as the ratio between the signal energy and the reconstruction error, is used as the performance metric to evaluate the quality of the reconstructed signal. MATLAB simulation results are presented 8 SNR(dB) = 20 log(||X||2 /||X − X̃||2 ), where X is the input signal and X̃ is the output of the algorithm 44 first and are followed by the experimental results from the hardware implementation. 2.9.1 Simulation results The finite time resolution tr of the TDC block in the ADC induces some quantization into the measurements. For the simulation experiments to follow, the quantization is kept at 7 bits (= log2 (ramp duration/tr ), with a ramp duration of 0.25 µsec and tr = 2 nsec). This corresponds to a signal to quantization noise ratio of about 44 dB for an input sinusoid. Multitone signals In the first experiment we reconstruct multi-tone input sig- nals, which are a linear combination of sinusoids. Each sinusoid has a random phase, comparable amplitude and its frequency is chosen randomly from the Nyquist grid. The Nyquist frequency is 3 MHz whereas the sampling frequency of the ADC is chosen to be 1 MHz, giving a sub-sampling ratio of 0.33. That is, K/N = 0.33, where K(= 150) is the number of measurements from the ADC and N (= 450) is the length of input signal, X. The input signal is corrupted by additive white Gaussian noise with varying power, sampled by the two sampling schemes and reconstructed using the PRSreco algorithm. The performance of the algorithms is evaluated by measuring the output SNR. The experiment uses the s-term Nyquist approximation as the benchmark performance, which is defined as the SNR obtained when the signal is sampled at Nyquist rate, quantized at the same quantization level as the ADC and then truncated, in frequency domain, to keep only the s dominant terms. The s-term Nyquist benchmark thus represents the best s-term approximation to the signal in frequency domain. Fig. 2.11(a) plots the mean (of 200 trials) reconstruction output SNRs for signals with 9 tones (corresponding s/N = 18/450 and s/K = 18/150) and 45 Figure 2.11: (a) Mean output SNR versus input SNR and (b) success percentage (fraction of trails that succeed) versus input SNR for 9-tone and 17-tone signals. The s-term NYQ (Nyquist) benchmark represents the best s-term approximation to the signal in frequency domain. Success means the correct identification of the frequencies of all tones. 17 tones (s/N = 34/450 and s/K = 34/150). The experiment demonstrates the better performance of random PPM in two ways. First, random PPM achieves a higher output SNR compared to the regular PPM and is closer to the benchmark9 performance, owing to the better correlation properties of the measurement matrix (Lemma II.1). The random PPM performance approaches the benchmark as the input SNR increases. Secondly, as the number of tones increases (making the signal less sparse), the random PPM output SNR is un9 The benchmark considers the error in the amplitude of the s tones due to quantization and input noise 46 affected relative to the benchmark while the output of constant PPM degrades. This indicates that the random PPM design can handle less sparse signals much better than the regular PPM scheme for same number of measurements. The output SNR can be higher than the input SNR, as the algorithm (like any other greedy pursuit algorithm) only calculates the coefficients of the top s frequencies in the signal and thus inherently filters out the noise at other frequencies. This “denoising” effect decreases as the value of s increases. This explains the degradation in output SNR (of even the benchmark) when the number of tones is increased. After input SNR is high enough, we see a saturation in the output SNR. This can be attributed to the quantization noise in the measurements (which also gets “denoised” to some extent). Fig. 2.11(b) plots the percentage of trials that achieve success in signal recovery. We call the reconstruction a success when the frequencies of all the tones in the input signal are correctly identified. Once again we observe that random PPM performs much better than the regular PPM. The plot also conforms that mean output SNR is a good indicator of the quality of reconstruction, as it also captures (to some extent) the information about the percentage of success. Sampling percentage The next experiment reconstructs a single tone signal (randomly chosen frequency, s/N = 2/450) with varying number of measurements and noise levels using the PRSreco algorithm. The sub-sampling ratio is defined as the ratio between the sampling rate of the ADC and the Nyquist rate of the signal (which is twice the randomly chosen tone frequency), and can be computed as K/N . The 47 Figure 2.12: Reconstruction of a single tone signal with varying number of measurements (a) with no noise (b) success percentage when no noise (c) sampling needed for 99% success, with noise sub-sampling ratio needed for at least 99% success (i.e. at least 99% of the total trials succeed in identifying the input signal frequencies correctly) is empirically determined for each input SNR level and is plotted in Fig. 2.12(c). We observe that at all SNR levels the random PPM ADC succeeds with far fewer measurements than the regular PPM. Further, when the input SNR is high enough the sub-sampling ratio needed for success in the random PPM quickly falls to about 3%. This can also be seen in the no-noise (i.e. only quantization noise) case (Fig. 2.12(a),(b)), where the regular PPM scheme breaks down when the sampling rate goes below 20% of Nyquist rate, whereas, the random scheme performs well enough for sampling rates 48 as low as 3% of the Nyquist rate, indicating much better incoherence properties of the measurement matrix. Figure 2.13: Reconstruction of a 11 tone signal with varying amount of time jitter noise Time jitter noise Noise generated in the ramp and comparator circuits degrades the accuracy of the time measurement. In this experiment, we model the time measurement error as a normal random variable with standard deviation σ. Fig. 2.13 plots the reconstruction results for a random 11-tone signal sampled by both the random and regular PPM ADCs, for varying σ (expressed as a multiple of the finite time resolution tr of the TDC block). While there is a degradation in the SNR performance of both the ADCs as σ increases, we see that the success percentage for the regular PPM ADC is much more sensitive to time jitter. Resolution versus sampling rate In this experiment we fix the bit-rate of random PPM ADC, that is the product of ADC quantization (resolution) and its 49 Figure 2.14: Mean output SNR versus random PPM ADC sampling rate, for fixed bitrates of 4, 5 and 7 Mbps. sampling rate. For example, a bit-rate of 5 Mbps can be achieved by choosing an ADC quantization of 5 bits and a sampling rate of 1 MHz. Fig. 2.14 displays the constant bitrate curves for bitrate values of 4, 5 and 7 Mbps for a random 11-tone input signal with input SNR of 15 dB. Each curve plots the mean output SNR for varying sampling rate. A low sampling rate corresponds to high ADC resolution and vice-versa (since the bitrate is fixed for each curve). If the sampling rate is too low, resulting in a lack of enough measurements, the reconstruction error increases, degrading the output SNR. If the sampling rate is too high, the output SNR again degrades due to lack of sufficient resolution in each measurement. This trade-off results in a sweetspot where the SNR performance is the best. From Fig. 2.14, we observe that this sweetspot occurs when the ADC resolution is chosen to be about 5 bits. 50 Figure 2.15: Output SNR vs input SNR for a demodulated FM signal FM signal with off-grid frequencies If a frequency falls in between two Nyquist grid points, it causes spectral spread or leakage, thus adversely affecting the sparsity of the signal. To counter this we propose to multiply the measurements from the ADC (before reconstruction) with a window function, like the Hamming (which is non-zero at all times, hence its effect can be reversed after the reconstruction). A frequency modulated (FM) signal with single tone message, where both the carrier and message frequencies are appropriately chosen to be off-grid acts as the input signal for this experiment. At 33% sampling, the noisy FM signal is windowed, reconstructed (using PRSreco algorithm), demodulated and the resultant output SNR is plotted in Figure 2.15. Some output SNR is lost as the amplitude of the message is smaller than the dynamic range of the ADC. The use of Hamming window improves the performance of the algorithm at all SNR levels and approaches the benchmark as SNR increases. Similar observations have been made for amplitude modulated (AM) signals with different message signals. The plots exhibit similar qualitative behavior when the sampling rate is increased or decreased. Note that for an FM 51 signal, windowing need not be reversed as the message is in the frequency of the signal. Figure 2.16: Output Vs Input SNR for a (a) multitone signal (b) demodulated AM signal with a sawtooth message Comparison of algorithms We now repeat the multi-tone (with 13 on-grid fre- quencies) signal reconstruction and the reconstruction of AM signal with sawtooth message (off-grid frequencies) experiments with the MOE algorithm, choosing the number of blocks m = 7. From the Fig. 2.16 we see that at low SNR conditions algo 2 (MOE) gives a better performance than algo 1 (PRSreco). This is because the identification stage in algo 2 (MOE) is more successful as it nullifies the effect of noise to some extent by taking the median over a set of m blocks. At high SNR, both the methods give comparable performance even though algo 2 (MOE) has only 1 iteration. Hence algo 2 (MOE) can be used to reduce computations whenever the input signal satisfies the additional conditions (in section 2.8), particularly if it is also known that the input SNR levels are low. We also observed that when Hamming window is employed the performance of algo 1 (PRSreco) improves whereas the algo 2 (MOE) shows little to no improvement. This is be- 52 cause, upon application of the Hamming window the input signal does not strictly satisfy the assumptions made in section 2.8 and hence the improvement in sparsity of the signal is balanced by the error amplification due to Hamming window reversal. 2.9.2 Prototype and measurement results We now present experimental results obtained with the prototype 9-bit random PPM ADC and 9-bit regular PPM ADC. The ramp generator, comparator and the two-step TDC, which are part of both the random PPM and the regular PPM ADCs, are implemented in 90 nm digital CMOS. The LFSR-based random clock generation block is implemented by programming Verilog code onto a Field-Programmable gate array (FPGA). The analog circuits operate with a 1 V supply, while the digital blocks operate at near-threshold from a 400 mV supply. The regular clock is a 64 MHz signal giving a Tclk = 15.63 nsec. The LFSR is 9 bits with taps at bin 5 and 9 resulting in a LFSR periodicity of 511. The entire evaluation setup of the random PPM ADC consists of four main blocks as displayed in Fig. 2.17, an FPGA, the ADC, a Logic Analyzer and a computer. The FPGA generates the start and the random clock signals, which are input to the PPM ADC. The ADC measurements are collected by the logic analyzer. The non-zero seed used to initialize the LFSR system is assumed to be known during reconstruction, so that the sequence of tRAN D ’s can be calculated. A single tone input signal is sampled both by the random PPM ADC prototype and the regular PPM ADC prototype, operating at various sampling rates, and reconstructed using the PRSreco algorithm. The results are displayed in Fig. 2.18. Also displayed for convenience is the compression loss (root mean square error of the reconstruction) on the right y-axis. Note that since the ADC resolution is fixed, 53 Figure 2.17: Hardware setup for the random PPM ADC the compression achieved by the sampling scheme only depends on the sub-sampling ratio K/N . As expected, the random PPM performs much better than the regular PPM which breaks down when the sub-sampling ratio is around 0.7, whereas the random PPM works well for sub-sampling ratios as low as 0.05. A compression ratio of 0.05 in the random PPM ADC and 0.7 in the regular PPM ADC, both result in the same compression loss of 0.77. Fig. 2.19 shows the reconstruction of a 5-tone signal with frequencies arbitrarily chosen from the Nyquist grid on [0, 1MHz] (Nyquist rate = 2 MHz). The multi-tone signal was sampled with the random PPM ADC operating at a sampling frequency of about 173 KHz which leads to a sampling percentage of about 8.65%. The SNR of the recovered signal is 41.6dB. The measured power consumption of the PPM ADC system is 14µW (excluding digital post-processing). The analog and digital blocks each consume 7µW. For the random PPM ADC system, the expected improvement in the power by a factor of K/N (the sub-sampling ratio) is observed, however this does not include the power consumed by the random clock generator and the comparator, both of which oper- 54 Figure 2.18: Reconstruction of a single tone signal from samples collected by the regular and the random PPM ADC prototypes operating at varying sampling rates. The y-axis on the right displays the corresponding root mean square (rms) error. ate all the time. The random clock signal is produced by the FPGA and the power consumption of the FPGA itself is not a good indication of the actual power needed since power is consumed by unnecessary circuitry in the FPGA. Implementing the random clock generation on the CMOS IC along with the rest of the compressive sensing ADC would only minimally increase the power consumption of the IC as the LFSR system only requires on the order of ten shift registers and a few gates. The PPM ADC itself uses approximately 50 registers and gates [1], therefore, the digital power consumption due to the addition of LFSR system, is expected to increase by only 6 − 8%. An additional power reduction can be achieved by switching the continuous-time comparator off, when not in use. Thus, the power consumption of 55 Figure 2.19: Reconstruction of a 5-tone signal from samples collected by random PPM with sampling rate at 8.65% of the Nyquist rate the random PPM ADC (with on-chip random clock generation) is estimated to be about (2 K + 0.07)7µW. For a random PPM ADC operating at 20% of the Nyquist N sampling rate (= 1 MHz), the estimated power consumption is 3µW. 2.10 Conclusion We propose a new low power compressive-sampling analog to digital converter, called the random PPM ADC. It inherits the advantages of time to digital conversion and also exploits compressive sampling techniques, to improve the power efficiency of data conversion. An existing PPM ADC design is modified to achieve a 9-bit random PPM ADC, through the use of a random clocking technique. The new random design enables the reduction of the average sampling rate to sub-Nyquist levels and thus reduces the ADC power consumption by a factor close to the sub-sampling ratio. 56 The random PPM performs much better than a regular PPM operating at a subNyquist sampling rate, in terms of obtaining closer-to-benchmark output SNR and handling signals that are less sparse. The proposed reconstruction algorithm is not only faster (greedy pursuit versus basis pursuit inspired algorithms in the literature for compressive sampling ADCs) but also feasible for a hardware implementation. With on-chip reconstruction and a low power front-end, the random PPM ADC is attractive for power constrained applications such as wireless sensor networks, as it reduces both the power consumption and the amount of data that needs to be communicated by each sensor node. 57 CHAPTER III Model Of A Sparse Encoding Neuron 3.1 Introduction Neurons as Time Encoding Machines (TEMs) have been proposed to capture the information present in sensory stimuli and to encode it into spike trains [51, 52, 53]. These neurons, however, produce spikes at firing rates above Nyquist, which is usually much higher than the amount of information actually present in stimuli. We propose a low-rate spiking neuron which exploits the sparsity or compressibility present in natural signals to produce spikes at a firing rate proportional to the amount of information present in the signal rather than its duration. We consider the IAF (Integrate-and-Fire) neuron model, provide appropriate modifications to convert it into a low-rate encoder and develop an algorithm for reconstructing the input stimulus from the low-rate spike trains. Our simulations with frequency-sparse signals demonstrate the superior performance of the Low-Rate IAF neuron operating at a sub-Nyquist rate, when compared with IAF neurons proposed earlier, which operate at and above Nyquist rates. It is a common belief that neurons encode sensory information in the form of a sequence of action potentials (nerve impulses or “spike trains”). The fundamental 58 unit of a “message” conveyed by a neuron is a single nerve impulse, propagating at high speed down its axon through well-understood electro-chemical processes [54]. These “spike trains” are interpreted by other neurons, leading to sensation and action. Fig. 3.1 illustrates a spike train produced by an auditory nerve cell. When we hear something, our brain is not actually interpreting the modulations in the acoustic waveform, but rather the spike trains generated, in response to the stimulus, by thousands of auditory nerves. In other words, spike trains form the language that the brain uses to communicate between neurons. Hence, understanding how a neuron encodes the stimulus or input signal into spike trains is of great interest. Figure 3.1: Spike trains produced by an auditory neuron Neurons generate spikes at relatively low rates, presumably due to a metabolic reason [55]. Metabolically efficient coding [56] is indicative of sparse encoding. Further, it has been observed that the process of spike encoding exhibits variability or randomness in response to identical inputs [57]. That is, for the same input stimulus, the neuron may produce different spike trains (as shown in Fig. 3.1). We are interested in developing a sparse encoding model of neuron that explains these observed 59 features of spike trains. Integrate-and-fire (or IAF, in short) models for neurons as generating time-stamp codes have been studied in [51] and [53]. Lazar et al. proved that a band-limited signal encoded by the precise spike timing of an IAF neuron can be reconstructed with reasonable accuracy from the spike train, when the average firing rate is above Nyquist rate [51]. When no other information is available about the input signal except its bandwidth, the signal has to be encoded at above Nyquist rate for successful recovery. However, most natural signals are often sparse or compressible in some orthonormal basis and hence the actual information present in the signal is usually much lower than the Nyquist rate. From an information theoretic point of view, a sparse encoding neuron should be able to encode such signals using spike trains that have a rate proportional to the amount of information actually present in the signals. In other words, most natural signals live in a low dimensional space and an efficient encoder should be able to capture the low dimensional information from the high dimensional signal. In this paper, we develop an efficient model of a sparse encoding neuron, which we call the Low-Rate IAF neuron, by performing appropriate modifications to a conventional integrate-and-fire model. The Low-Rate IAF neuron exploits the sparsity or compressibility of input signals to encode them into spike trains with rates well below the Nyquist rate. We show that the low-rate spike trains contain enough information about the input stimulus to allow its recovery and develop a neural decoding algorithm based on spike times. 60 The remainder of the chapter is organized as follows. The input signal model is described in Section 3.2. A relevant background on time encoding through integrateand-fire neurons, including the model proposed by Lazar in [53], is briefly presented in Section 3.3. The proposed Low-Rate IAF neuron is presented in Section 3.4 and is followed by a description of the reconstruction algorithm in Section 3.5. A set of numerical experiments compare the performance of Lazar’s IAF neuron (from [53]) with the proposed Low-Rate IAF neuron in Section 3.6. We conclude with a discussion on future work in Section 3.7. 3.2 Input stimulus model The class of input signals is assumed to be band-limited with cutoff frequency W (in Hz) and periodic within a time period D. The Nyquist rate of the input signal space is thus FN = 2W . W and D are related by W = N 2D where N is a positive integer that denotes the dimension of input space. If an input signal/stimulus x(t) is sampled at Nyquist rate for a time duration of D, then the number of samples obtained is N = FN D. Thus the signal x(t), observed for a time duration D, can be represented as a vector x of length N in discrete domain, where x[i] = x(i/FN ) for i = 1, .., N . The signal x(t) is further assumed to be S-sparse or compressible in frequency domain. A signal is called S-sparse in the frequency domain, if the DFT (discrete Fourier transform) of the signal samples at Nyquist rate has only S non-zero terms. That is, if X represents the DFT of vector x, then X has at most 61 S non-zero elements. A signal is called S-compressible1 in frequency domain, if the sorted list of its DFT coefficients has only S significant or dominant terms, compared to which the other terms are negligible. Thus, a compressible signal is one that is reasonably well approximated as a sparse signal. The input signal can be expressed as a linear combination of complex exponentials as follows: x(t) u S X cm exp(j2πfm t) m=1 where fm , m = 1, .., S are the S dominant frequencies which lie in the interval [−W, W ] and cm are the corresponding coefficients. We further assume that the input signal is real-valued, hence S is even and one set of frequencies are the negative of the other set. Thus, the input stimulus is a mixture of periodic waveforms, which is consistent with the brain mechanism of generating and entraining oscillations at multiple frequencies simultaneously. 3.3 Time encoding with Integrate-And-Fire Neurons In this section we review the time encoding machine (TEM) consisting of an integrate-and-fire (IAF) neuron [51, 52, 53]. Neurons encode continuous time sensory stimuli into discrete time events, i.e. the firing of action potentials at variable time points. Time encoding is an answer to one of the key questions arising in information processing, which is, how to represent a continuous signal as a discrete sequence. In conventional sampling, a band-limited signal is represented by set of amplitude samples spaced uniformly. If the uniform spacing is chosen to satisfy the 1 We call X, S-compressible, if it is well approximated as a S-sparse signal, ||X − X −α for some (S) ||2 ≤ C · S constants C and α > 0, where X(S) is the S-sparse signal that best approximates X. 62 Nyquist rate condition, the signal can be recovered perfectly, under no noise, through sinc interpolation. This is the well-known Shannon sampling theorem. In contrast, time-encoding of a real-valued band-limited signal is an asynchronous process of mapping the amplitude information into a strictly increasing sequence of time points. A time encoding machine (TEM) is a realization of such encoding. The reconstruction of input signal from the sequence of time points is referred to as time decoding. 3.3.1 Preliminaries A typical IAF TEM neuron is schemtaically shown in Fig. 3.2. A constant bias b (b > 0 such that x(t) + b > 0, ∀t) is added to the input signal, which is then fed to the integrator. When the output of the integrator crosses a threshold δ, a spike is produced. The spike triggers a zero reset of the output of the integrator. The output of the TEM is thus a sequence of spikes at time points, {tk }, that models the spike train produced by a neuron. Figure 3.2: Time encoding with an integrate-and-fire (IAF) neuron Let K denote the number of spikes produced by the IAF neuron in the duration D for which the input stimulus is observed. From simple calculations we can easily 63 derive, Z tk+1 x(s) ds = κδ − b(tk+1 − tk ) tk for k = 0, .., K −1, where t0 is the time point at which we begin to observe the signal. If |x(t)| ≤ c, ∀t, then the inter-spike-interval is bounded by, κδk κδ ≤ tk+1 − tk ≤ b+c b−c It has been proved [51][52] that a successful recovery of x is possible when, κδ 1 < b−c 2W that is, the maximum inter-spike-interval is smaller than the Nyquist period TN = 1/FN = 1/2W . Hence, the TEM IAF neurons encode all input signals at an average rate greater than the corresponding Nyquist rate. 3.3.2 Integrate-And-Fire Neurons with Random Thresholds To model the variability or randomness characteristic of neuronal spike trains, neurons with random thresholds were proposed in [58]. An IAF neuron model with random thresholds is studied by Lazar in [53]. The model is identical to the TEM shown in Fig. 3.2, but with random thresholds δk . Every output spike not only resets the integrator output but also triggers the random selection of a new threshold δk . The random thresholds are assumed to be drawn from a Gaussian distribution with known mean δ and variance σ 2 . For random thresholds TEM, let us define a measurement vector q and error vector ε of length K, as follows. For k = 0, .., K − 1, qk = κδ − b(tk+1 − tk ), 64 εk = κ(δk − δ). Time-encoding can be expressed as the following system of equations, GX = q + ε where X is the N -point DFT of vector x and G (of size K x N ) is given as Z tk+1 Gk,n = n ej2π N FN s ds tk for k = 0, .., K − 1 and n = −N/2, .., N/2 (assuming N is even and with a slight abuse of notation). A weighted least squares with `2 penalty is used for reconstructing an approximation X̃ of X from q X̃ = argmin||q − GX||2 + Kλ||X||2 . Here, λ is a positive smoothing parameter that regulates the trade-off between faithfulness to the measurements and smoothness. The regularization is used to prevent over-fitting due to the noisy data. For a successful recovery, the method requires that the average spike rate be above Nyquist rate [53]. In other words, we need the number of spikes K > N . Note that N is the number of samples at Nyquist rate and hence Lazar’s TEM neuron is firing at rates above Nyquist. In the next section we develop a low-rate model of IAF neuron that fires at a sub-Nyquist rate. 65 3.4 The Low-Rate Integrate-and-Fire Neuron We introduce appropriate modifications in the TEM IAF neuron and develop a low-rate IAF neuron model. The Low-Rate IAF neuron schematical is shown in Fig. 3.3. We use fixed thresholds (δ) as opposed to random thresholds used in Lazar’s model. The randomness in inter-spike-interval exhibited in neuronal spike trains is produced by an additional component that switches off the IAF circuit for a random amount of time τk (with mean µ) after each spike (see Fig. 3.3). The process of switching off the IAF circuit mimics the “absolute refractory” period exhibited by a neuron. After a single impulse, a dormant period occurs during which no other impulse can be initiated [54], which is called the “refractory” period. We model this refractory period as a random variable to account for the randomness in neuronal spike trains in response to identical inputs. Figure 3.3: Sparse time encoding with Low-Rate integrate-and-fire(IAF) neuron. The time durations τk are assumed to be uniformly distributed with mean µ. The operational equation of time-encoding can be obtained as follows, Z tk+1 x(s) ds = κδ − b(tk+1 − tk − τk ) tk +τk for k = 0, .., K − 1. Similar to Section 3.3, we define measurement vector q and 66 matrix G as follows, qk = κδ − b(tk+1 − tk − τk ) Z tk+1 n ej2π N FN s ds Gk,n = tk +τk for k = 0, .., K − 1 and n = −N/2, .., N/2. In an actual implementation of the Low-Rate IAF neuron in hardware, the time durations τk can be generated using a pseudo-random number generator such as linear feedback shift register (LFSR). If the seed that is used to initialize the LFSR is assumed to be known, then τk can be computed by the reconstruction algorithm. An alternative is to actually measure τk using a time to digital converter (TDC). The measurements qk can thus be computed by the reconstruction algorithm. The low-rate neuron produces spikes at a sub-Nyquist rate determined by the parameters δ and µ. Let K denote the number of spikes produced in duration D, then K < N . We are interested in solving for X (the N -point DFT of input signal) given tk for k = 0, .., K − 1, i.e., we want to solve the following linear system of equations for the case when K < N , GX = q + ξ where ξ is a noise vector, which can model additive noise at the input or a time jitter noise in measuring tk . The problem is ill-posed in general, since it is underdetermined and has infinitely many solutions. But under the assumption that X is sparse or compressible (as described in Section 3.2), it may be possible to uniquely recover X. We develop a new recovery technique to reconstruct X, which is described in the next section. 67 3.5 The Reconstruction Algorithm Given the measurements q = GX + ξ of a sparse or compressible signal X (of length N ), with number of measurements K < N , the novel area of Compressive Sensing (CS) offers explicit constructions or distributions on matrices G and algorithms such as those proposed in [8],[7] and [3], to recover an approximation of X (denoted by X̃). One line of research assumes that the measurement matrix G satisfies a property called the restricted isometry (RIP) [3], and uses either greedy iterative algorithms ([8],[7],[6]) or convex optimizations to obtain X̃. Another line of research designs matrices G and algorithms jointly, optimizing for reconstruction time [13], storage requirements of G, or physical realizability [16] of measurement with matrix G. The matrix G produced by an IAF time-encoding system (whether deterministic or random) does not necessarily satisfy the RIP condition. Hence, following the second line of research, we co-designed the measurement system (i.e. the Low-Rate IAF neuron model) and the recovery algorithm, keeping in mind the physical realizability of the TEM as well as the TDM (time decoding machine). In this section, we describe the reconstruction algorithm developed for the Low-Rate IAF neuron model presented in Section 3.4. We begin by transforming GX = q into a new system of equations BX = y by doing the following. From mean value theorem, we know that there exists sk ∈ (tk + τk , tk+1 ) such that Z tk+1 x(sk )(tk+1 − tk − τk ) = x(s) ds. tk +τk Thus we can define sk for k = 0, .., K − 1 and the corresponding signal amplitudes 68 as yk = x(sk ) = qk . (tk+1 − tk − τk ) We define a new measurement vector y in this manner. The N -point DFT X and measurement vector y can be related as BX = y, where the new measurement matrix B (of size K x N ) is given by n Bk,n = ej2π N FN sk for k = 0, .., K − 1 and n = −N/2, .., N/2. Note that B is not really a sub-DFT matrix, since s0k s do not have to lie on a Nyquist time grid. A pseudo-code of the reconstruction algorithm is presented in Table 3.1. For a vector z, supp(z) is defined as the set of indices of the non-zero elements of z and z(s) stands for the best s-term approximation2 of z. For an index set T ⊂ {1, 2, .., N }, zT stands for a sub-vector of z containing only those elements of z that are indexed by T . Similarly GT stands for a sub-matrix of G containing only the columns of G indexed by T . The matrix B is similar to the matrix used in [16] and hence we use the algorithm developed in [16] to estimate the indices of the dominant terms in X, that is, we identify the dominant frequencies in X. The largest components in B H y provide a good indication of the largest components in X [16]. The algorithm applies this idea iteratively to reconstruct an approximation to the signal X. At each iteration, 2 The best s-term approximation of a vector z can be obtained by equating all the elements of z to zero, except the elements that have the top s magnitudes. 69 the current approximation induces a residual, which is the part of the signal that has not been approximated yet. The current approximation vector X̃ is initialized to a zero vector and the residual is initialized to the measurement vector y. At the end of iterations, once the dominant frequencies are identified (denoted by index set T in Table 3.1), their coefficients (i.e. the elements of XT ) are then estimated through performing a least squares with a truncated matrix GT . We approximate sk = (tk+1 + tk + τk )/2. The reconstruction algorithm input: N (signal length), S (sparsity), (sk , yk ),k = 0, 1, .., K − 1. output: X̃ (S-sparse approximation to X, length N ) X̃ (0) = 0, residual r(0) = y for i = 0, 1, 2, .. X̃ (i+1) = [X̃ (i) + B H r(i) ](S) r(i+1) = y − B X̃ (i+1) until ||r(i+1) ||2 does not vary within a tolerance θ. n o T =supp X̃ −1 H GT y X̃T = GH T GT X̃T c = 0 (Least Squares) Table 3.1: The Reconstruction Algorithm The computationally intensive step of least squares is performed only once in the algorithm. The least squares is implemented using the accelerated Richardson iteration [48] with runtime of O(SKlog(2/et )) where et is a tolerance parameter. The structure of the measurement matrix lends us to use the inverse NUFFT [49] with cardinal B-spline interpolation for forming the products of the form B H r, in a runtime of O(N logN ). Hence the total runtime of the algorithm is dominated by O(IN logN ) where I is the number of iterations which has a gross upper bound of 70 logN . In practice, we find that the approximation sk u (tk+1 + tk + τk )/2 is good when the threshold δ is small enough. It is possible to update sk , k = 0, .., K − 1 using the current approximation X̃ at the end of each iteration, by using Newton’s method for example. More sophisticated methods might yield better results. 3.6 Results and Discussion Lazar’s TEM neuron and our Low-Rate IAF neuron are simulated in MATLAB, along with the reconstruction algorithms. We compared the performance of our LowRate neuron firing at sub-Nyquist spike-rate with TEM neurons in [53] operating at and above Nyquist rate. We define the sparse-encoding ratio of Low-Rate IAF neuron as K , N which implies that the firing rate of the neuron is K F . N N The input signal, as explained, is assumed to be a mixture of sinusoidal waveforms of S frequencies. Because we inject additive white Gaussian noise into the input signal, we use the traditional measure of signal-to-noise ratio (SNR) as the performance metric. The output SNR3 is defined as the ratio between the signal energy and the reconstruction error, whereas the input SNR is defined as the ratio between signal energy and noise energy. In the first experiment, we choose S = 10 and compare the recovery performance of Lazar’s TEM neuron and Low-Rate IAF neuron. The sparse-encoding ratio of Low-Rate neuron is chosen as K/N = 0.3052. Fig. 3.4 plots the mean output SNR vs. input SNR. We see that the Low-Rate IAF neuron (even when operating at about one third the Nyquist rate in this example) outperforms the TEM neurons 3 Output SNR(dB) = 20 log(||X||2 /||X − X̃||2 ), where X is the input signal and X̃ is the output of the algorithm 71 Figure 3.4: Output SNR vs input SNR for signals with S = 10 (which are not sparse encoders) operating at and above Nyquist rates. Moreover, we see that Lazar’s reconstruction degrades significantly when the average firing rate of TEM neurons is reduced to about 0.97FN . In the next experiment, we choose S = 60. Mean output SNR vs. input SNR is plotted in Fig. 3.5 for Low-Rate IAF neuron operating at different rates and Lazar’s TEM neuron operating at about twice the Nyquist rate. To match the performance of Lazar’s TEM neuron at twice the Nyquist rate, we need to set the firing rate of the Low-Rate IAF neuron to about 0.38 times the Nyquist rate. Fig. 3.5 demonstrates that an increase in sparse-encoding ratio K/N improves the performance of the Low-Rate IAF neuron. 72 Figure 3.5: Output SNR vs input SNR for signals with S = 60 3.7 Conclusion and Future Work We proposed a model for a sparse encoding neuron, called the Low-Rate IAF (integrate-and-fire) neuron, which is an adaptation of the TEM IAF model proposed by Lazar [51, 52, 53]. Lazar’s TEM model produces spikes above Nyquist rate, which is usually much higher than the amount of information actually present in the input sensory stimuli. By exploiting the sparsity, the Low-Rate IAF neuron encodes input stimulus into spike trains with average firing rate well below Nyquist rate, while using the spike timing information in a smart manner to improve the performance of stimulus recovery. The developed reconstruction algorithm is computationally efficient and can be tailored for practical hardware implementations. A number of other time-encoding neuron models, including many other IAF architectures, have been proposed in the literature. The methodology of low-rate or sparse encoding, along with the developed reconstruction algorithm, can be extended to these neuron 73 models. This direction will be explored in the future. We are also interested in investigating the application of our Low-Rate neurons in developing a sparse encoding model for videos. The classification of input stimuli from low-rate spike trains is another potential future direction. 74 CHAPTER IV Continuous Fast Fourier Sampling 4.1 Introduction The problem of quickly computing the largest few Fourier coefficients of a signal from a given (sliding) time window arises in numerous situations. For example, in cognitive radio [59], where a wireless node alters its transmission or reception parameters based on active monitoring of radio frequency spectrum at various times, or in incoherent demodulation of communication signals [60] (such as FSK, MSK, OOK, etc.,) where the computed frequency spectrum at different times represents the message being transmitted itself. Other applications include data compression, feature extraction, data mining, continuous monitoring of signals, real-time change detection in signal parameters, etc. Most of these applications involve large signal sizes or bandwidths while the signal is often redundant (sparse or compressible), with only a few Fourier coefficients that are of interest. In such cases the Fast Fourier Transform (FFT), which computes all the Fourier Transform (FT) terms, is computationally wasteful. Hence algorithms with efficient storage requirements and low runtime are of primary importance. Moreover, the resource efficient algorithm should be able to quickly analyze a signal from any arbitrary placed time window. 75 Compressed Sensing (CS) methods [3] [6] provide a robust framework for reducing the number of signal samples required to estimate a signal’s Fourier transform. Although the storage requirements are small, standard CS Fourier methods often utilize Basis Pursuit (BP) [3] and greedy matching pursuit algorithms [6] that have a runtime super-linear in signal’s size/bandwidth, and hence, inappropriate for applications such as those described above. A second body of work on algorithmic compressed sensing includes methods which focus on achieving near-optimal running times [61] [62]. However these algorithms do not achieve sub-linear storage requirements. Fourier sampling algorithms [11] [63] achieve both sub-linear storage and runtime requirements by exploiting the spectral redundancy of signals. In particular, a randomized Fourier sampling algorithm called the AAFFT (Ann Arbor Fast Fourier Transform) [63] has been shown to outperform the FFT in terms of runtime while utilizing only a fraction of the FFTs required samples [64]. In these algorithms, unevenly spaced samples of the signal (from a given time window) are acquired in a structured random fashion, below Nyquist rate. These samples are used in a nonlinear iterative manner to quickly estimate the signal’s dominant Fourier coefficients. The structure in the random sampling pattern, however, depends upon the boundaries of the time window in which the signal is analyzed (see Section 4.2.2). Thus an arbitrary placing of the analysis window is not accommodated. We propose the Continuous Fast Fourier Sampling (CFFS) algorithm which is both a highly efficient reconstruction algorithm (like AAFFT) and adapted for arbitrary sliding window calculations, thus attractive for the mentioned applications of interest. 76 The AAFFT algorithm and its limitations are briefly discussed in Section 4.2.2. The CFFS algorithm is described in detail in Section 4.3, followed by theorems that prove its correctness in Section 4.3.3. Section 4.4 presents a few results and numerical experiments that provide proof of concept and apply the CFFS algorithm to decoding frequency hopping signals with known and unknown change points. 4.2 Background and preliminaries The algorithms in this chapter and their analysis are inherently discrete. The samples are drawn from a discrete time signal (rather than an underlying continuoustime signal) and output of the algorithms is an approximation to the discrete Fourier spectrum of the signal. 4.2.1 The problem setup and notation Let the input discrete time signal be denoted by x of length n (n very large). Let y denote the signal x from a given analysis window or block of length N (N << n and N = 2α for some integer α). If (n1 , n2 ) are the boundaries of the analysis window, then n2 − n1 + 1 = N and y(i) = x(i − n1 + 1), for i = 1, .., N . y is assumed to be sparse or compressible in the frequency domain. A signal is called m-sparse in frequency domain, if its Discrete Fourier transform (DFT) has only m non-zero terms, while it is called m-compressible in frequency domain, if the DFT has m dominant coefficients with other negligible coefficients. So y can be viewed as superposition of m dominant frequencies. An algorithm is called sub-linear if it has O(m poly(log(N )) runtime and storage requirements. Furthermore, an algorithm is called “continuous” or “sliding window algorithm” if it can accommodate arbitrary positions of block y. 77 We develop the CFFS algorithm (section 4.3), which is a sub-linear sliding window algorithm. 4.2.2 The Ann Arbor Fast Fourier Transform (AAFFT) The AAFFT is predicated upon non-evenly spaced samples (from block y), unlike many traditional spectral estimation techniques [65, 66] and uses a highly nonlinear reconstruction method that is divided into two stages, frequency identification of the m dominant frequencies and coefficient estimation, each of which include multiple repetitions of basic subroutines. A detailed description of the implementation of AAFFT is available in [63]. Figure 4.1: Figure showing the samples acquired in AAFFT for each (t, σ) pair Frequency Identification consists of two steps, dominant frequency isolation and identification. Isolation is carried out by a two-stage process: (i) random time dilation of y (corresponds to a random permutation of the spectrum of y), followed by (ii) the application of a filter bank with K = O(m) filters. The probability of isolation of dominant frequencies by different filters is increased with repetitions. Note that all the above is carried out conceptually in the frequency domain but instantiated 78 in the time domain. In each repetition, a pair (t, σ) is chosen randomly with t ∼ U[1, 2, .., N ] and σ ∼ U[1, 3, .., N − 1] and the samples of the signal block y indexed by the matrix in Fig. 4.1 are used to perform computations. Let P (t, σ) = {(t + qσ) mod N, q = 0, 1, .., K − 1} be the arithmetic progression that forms the first row in figure 4.1. The other rows consist of arithmetic progressions P (tb , σ), where tb is an element of the geometric progression tb = t + N , 2b+1 b = 0, 1, .., α − 1. The isolation stage performs K-point FFT along each row of the matrix. After the FFTs, the ith column contains the output of ith filter in the bank, evaluated at time points t, t + N/2, t + N/4, ... given by the above geometric progression. The identification stage performs group testing across each column to determine the (bits of the binary representation of the) dominant frequency isolated by the corresponding filter. Let A1 = {(t, σ)} be the set containing all the (t, σ) pairs used in the frequency identification stage. Similarly, let A2 be the set containing the (t, σ) pairs used in the estimation stage (which also uses the random sampling pattern similar to the first row of figure 4.1, for coefficient estimation of each of the identified dominant frequencies). The whole process takes time and storage in the order of mpoly(log(N )). Note that although the (t, σ) pairs in A1 and A2 are chosen randomly, the sample indices that result from each pair are highly structured. Moreover, the indices are dependent on the boundaries of block y (due to the mod N arithmetic). Thus AAFFT can analyze the input signal x by dividing it into consecutive non-overlapping blocks or windows of length N . Let us call this block-based analysis method S1. S1AAFFT clearly cannot accommodate arbitrary position of analysis window. This is illustrated in Fig. 4.2 for a simple case of N = 32 with a dummy y-axis for clarity. The X’s represent the indices where the samples are acquired by the S1-AAFFT 79 procedure from two consecutive blocks B1 and B2. The O’s represent the indices where the samples are needed for applying AAFFT to an arbitrarily chosen block B. As can be seen in the figure, the S1-AAFFT procedure did not acquire all the O’s. Figure 4.2: 4.3 Figure showing the samples acquired by S1 (X’s) and the samples (O’s) required to apply AAFFT on B = [16, 47] Continuous Fast Fourier Sampling In this section we construct a new sampling procedure for signal x, called the CFFS, that permits a fast reconstruction algorithm (like AAFFT) on arbitrarily placed analysis windows of length N from signal x. 4.3.1 Sample set construction For each (t, σ) pair, define a sequence of time points t(j), j = 1, .., J (with t(0) = t and J = d Kσ e) such that t(j) is the “N -wraparound” of t(j − 1). FigN ure (4.3) illustrates the calculation of a N -wraparound. Mathematically, t(j) = (t(j − 1) + Q(j − 1)σ)modN where Q(j − 1) is the smallest integer such that 80 t(j − 1) + Q(j − 1)σ ≥ N . Figure 4.3: Calculation of N -Wraparound t(1) from t For j = 1, .., J, denote by Ij the following arithmetic progression formed by (t(j), σ), Ij = {t(j) + qσ, ∀q ≥ 0 : t(j) + qσ ≤ n} (4.1) N for b = 0, 1, .., α − 1. For Now, consider the geometric progression tb = t + 2b+1 N each b, t + 2b+1 , σ is treated as another (t, σ) pair and the sequence tb (j) and the corresponding progressions Ijb can be calculated. For each pair (t` , σ` ) in A1 and A2 , expand as above and denote the arithmetic progressions produced by I`,j , for S` I`,j . I` j = 1, .., J` . Define the union of all such arithmetic progressions as I` = Jj=0 is shown in Fig. 4.4. Figure 4.4: Figure showing the arithmetic progression samples acquired in CFFS for a (t` , σ` ) pair and their wraparounds 81 Similarly define I`b = S J` b j=0 I`,j Now define the union I`B = (4.2) for each b = 0, .., α − 1. Sα−1 I`b . Finally define ! [ I(A1 , A2 ) = (I` ∪ I`B ) ∪ b=0 A1 ! [ I` . A2 Given a set of indices I, we denote by S x (I) the set of samples from signal x indexed by I. 4.3.2 The CFFS Algorithm Preprocessing input: N // Block length (1) Sample-set generation : Choose A1 and A2 as defined and compute I(A1 , A2 ) (as in Equation (4.2)). output: I(A1 , A2 ) // Index set Sample Acquisition input: I(A1 , A2 ), x (2) sample signal x at I and obtain samples S x (I). output: S x (I) Reconstruction input: S x (I), (n1 , n2 ) // boundary indices of an arbitrary block y of length N from signal x (3) calculate A01 , A02 (defined in Section (4.3.3)) and extract S y (I(A01 , A02 )) ⊂ S x (I). (4) apply AAFFT on the sample-set S y (I(A01 , A02 )) output: top m frequencies of x in block y = x[n1 , n2 ] Table 4.1: The Continuous Fast Fourier Sampling (CFFS) algorithm 82 4.3.3 Proof of Correctness of CFFS In this section we show that CFFS permits application of AAFFT on any arbitrarily placed block y in signal x. We define new sets A01 and A02 as follows. Put A01 = {(t0 , σ) : (t, σ) ∈ A1 }, where t0 is the n1 -wraparound of t. Mathematically, t0 = (t + iσ)modn1 where i is the smallest integer such that t + iσ > n1 . Similarly define A02 . Note that A01 and A02 are still random since A1 and A2 were chosen randomly. AAFFT is applied on y with the sampling pattern defined (in Section (4.2.2)) from A01 and A02 . The following theorems together show that the required samples of y are available in S x (I(A1 , A2 )). Theorem IV.1. For sets A01 and A02 as defined above, S y (I(A01 , A02 )) ⊂ S x (I(A1 , A2 )). Theorem IV.2. AAFFT can be applied using the sample-set S y (I(A01 , A02 )), i.e. the index set I(A01 , A02 ) has the required structure explained in Section (4.2.2). Rather than giving detailed proofs, we prove a proposition that lies at the heart of the two theorems. Proposition IV.3. For every (t0 , σ) in A01 or A02 , S y (P (t0 , σ)) ⊂ S x (I(A1 , A2 )). Proof. Let (t, σ) be the pair in A1 or A2 from which (t0 , σ) was obtained. We will prove that the arithmetic progressions Ij formed by the sequence of wraparounds t(j),j = 1, .., J as defined in Section (4.3.1), induce mod-N arithmetic in the progression P (t0 , σ) (P as defined in Section (4.2.2)). Consider the first few terms in P (t0 , σ), till (t0 + (q0 − 1)σ) mod N where q0 is the smallest integer such that (t0 + q0 σ) ≥ N . From definition of t0 observe that t0 = (t + iσ − n1 ). so y(t0 ) = x(n1 + t0 ) = x(t + σ) ∈ S x (I0 ), 83 where I0 is defined in Equation (4.1). Similarly it is easy to see that the first q0 terms in S y (P (t0 , σ)) are contained in S x (I0 ). Now call the next term (t0 + q0 σ) 0 mod N = t0 (1). Observe that t0 (1) = t0 + σ N σ−t − N . Similarly observe that t(1) = t + σ Nσ−t − N . Now, Substituting t0 = (t + iσ − n1 ) in the expression for t0 (1) we get, N − t + n1 − iσ t (1) = t + iσ − n1 + σ −N σ N −t = t + iσ − n1 + σ + dσ − N σ 0 = t(1) + (i + d)σ − n1 , for an appropriately defined d, which can be shown to be positive. So, y((t0 + q0 σ) mod N ) = y(t0 (1)) = x(t(1) + (i + d)σ) ∈ S x (I1 ), where again I1 is defined in Equation (4.1). Let q1 be the smallest integer such that (t0 (1) + q1 σ) ≥ N . Now it is easy to see that the next q1 terms in S y (P (t0 , σ)) are contained in S x (I1 ). Repeat this until all the terms in P (t0 , σ) are covered. Proposition IV.4. On average, the storage requirement of CFFS algorithm is in the order of O( Nn m logO(1) N ), which is of the same order as that of a sampling scheme which divides the signal into n/N non-overlapping blocks and samples each block for AAFFT. 84 Figure 4.5: 4.4 The Sparsogram (time-frequency plot that displays the dominant frequencies) for a synthetic frequency-hopping signal consisting of two tones. The same sparsogram is obtained both by AAFFT (S1) and CFFS Results and Discussion The Continuous Fast Fourier Sampling algorithm has been implemented and tested in various settings. In particular, we performed the following experiments. Frequency hopping signal with known block boundaries: we consider a model problem for communication devices which use frequency-hopping modulation schemes. The signal we want to reconstruct has two tones that change at regular intervals which are assumed to be known. The signal is assumed to be noiseless. We apply both the straightforward S1-AAFFT and CFFS to identify the location of the tones. Figure (4.5) shows the obtained sparsogram which is a time-frequency plot that displays only the dominant frequencies in the signal. We get the same sparsogram in both cases, as expected. S1-AAFFT samples about 0.94% of the signal whereas CFFS samples about 1.06% of the signal, which is only very slightly larger than S1. This experiment demonstrates the efficiency and similarity of the 85 two methods and supports the proposition made in Section (4.3.3). Figure 4.6: Applying CFFS to different blocks of signal x Arbitrary position of analysis window: While S1-AAFFT cannot be applied to compute the dominant tones in any arbitrary block, the CFFS has no such limitation. This is demonstrated in the next experiment as follows. Let y be a signal of length N = 220 , with m = 4 known dominant frequencies. Note that the specific values of m and N are not integral to the performance of the algorithm. AAFFT has been tested exhaustively and its performance as a function of N ,m is completely characterized [24]. Let x be an arbitrary signal of length n with N n. Now let x[n1 , n2 ] be an arbitrary block of interest of length N . Set x(n1 + q) = y(q), for q = 0, 1, . . . , N − 1. Thus we have placed a copy of the known signal y in the block of interest. The CFFS was then applied and the four dominant frequencies in the block of interest were computed. The obtained values for frequencies and their coefficients match closely with those of the signal y and satisfy the error guarantees of AAFFT. The whole experiment was repeated with different values for n1 (and corresponding n2 = n1 + N − 1) and the same results were obtained. Figure (4.6) shows the sketch 86 of a signal x, pre-sampled in a predetermined manner (according to CFFS), with copies of y placed at arbitrary positions. Applying AAFFT to any block with a copy of y gives the same results thus demonstrating the correctness of CFFS. Unknown Frequency hopping signal: For simplicity let’s assume that the signal has two tones that change at certain intervals which are not known. We are interested in finding the unknown boundaries at which the tones change. In particular, consider two adjacent blocks with f1 and f2 as their respective frequencies (see Figure (4.7)). We take an analysis window of length N . The center of the window can be varied and a “binary” search can be performed for the block boundary in the following manner. If the center is to the left of the actual boundary, then the coefficient of f1 will be higher than that of the f2 . This indicates that the center has to be moved to the right from its current position. This step can be iterated a few times to make the center converge to the actual block boundary. Figure 4.7: Frequency-hopping signal with unknown block boundaries. Note that looking at the relative ratios is just one way (albeit simple) of doing change point detection in the frequency domain. More sophisticated algorithms ex- 87 ist ([67],[68],[69]), however they are not designed to work on sub-Nyquist samples and are computationally intensive (with runtime superlinear in signal length). The signal is sampled using the CFFS pattern, thus enabling the application of AAFFT on any analysis window of length N while performing the search. Also the search is not strictly binary since the amount by which f 1 coefficient is higher than f 2 can be used to shift the center of the window to the right by an equivalent amount. Once the center converges after a few iterations, we express the error as the distance to the true boundary and determine what percentage of the block this distance is. Table (4.2) displays the error and how the error increases with decreasing SNR. SNR(dB) ∞ 10 8 6 4 2 %Error 0.39 0.58 0.70 0.78 0.79 1.56 Table 4.2: Percentage error in boundary identification Note that even in the case of no noise (infinite SNR) there is some inherent ambiguity in the identification of block boundary. This uncertainty is caused by two factors. First, when the analysis window has portions of both the f 1-block and f 2-block, the net signal is no longer sparse due to a sudden change in frequency and has a slowly decaying spectrum. With m = 2 the AFFT guarantees that the error made in signal approximation is about as much as the error in optimal 2-term approximation [63]. Hence a slowly decaying spectrum implies more error in the approximation. A second and more important factor is the number of samples actually acquired from the region of uncertainty around the block boundary. From the 88 entire block, CFFS acquires about 8% samples from the N = 217 present. Assuming these samples are uniformly distributed (which is not true for CFFS), the number of samples present in the region of uncertainty (0.4%) is about 40. In practice, CFFS contains even fewer samples in the uncertainty region (about 30 on average). In terms of samples actually acquired in CFFS, the boundary estimation is off by only a few samples and hence is negligible, as it does not affect the computations. This will be true for any sparse sampling method like CFFS. Furthermore, if the uncertainty were to be reduced to 0.3% say, the boundary identification would improve by only about 6 samples on average, which again is negligible. Hence the boundary identification through the above method is accurate enough for all practical purposes. 4.5 Conclusion and Future Work We described and proved a sub-linear time sparse Fourier sampling algorithm called the CFFS which along with AAFFT can be applied to compute the frequency content of sparse digital signals at any point of time. Once the block length N is selected, a sub-Nyquist sampling pattern can be pre-determined and the samples can be acquired from the signal (during the runtime if required). The AAFFT can be applied to the samples corresponding to any block of length N of the signal and the dominant frequencies in that block and their coefficients can be computed in sub-linear time. The algorithm requires the block length N to be fixed beforehand. Designing or extending the algorithm to work for different values of N can be considered in the future. Adapting the algorithm to further reduce the computational complexity by using known side information about the signal can also be considered. The algorithm is also highly parallelize-able and can be adapted for hardware 89 applications. Also, we may be able to extend this sample set generation to the deterministic sampling algorithm described in [12] and the sparse FFT algorithm in [14]. 90 CHAPTER V Spectrum Sensing Cognitive Radio 5.1 Introduction In recent years, as a result of numerous emerging wireless applications and services, a scarcity in spectral resources and an increased demand for available spectrum has been witnessed. This scarcity, however, is paradoxical since most of the allocated spectrum remains underutilized at any given time and geographic location. This paradox occurs due to the static nature of the current spectrum licensing scheme which allocates the channels or bands of the spectrum to the primary (licensed) users (e.g. TV broadcast channels, mobile carriers), who do not transmit at all times and locations. This results in spectrum holes or vacancies. These spectral holes are thus free to be used by unlicensed or secondary users. Exploiting this fact, a Cognitive Radio (CR) is proposed in [70]. A CR is a “smart” radio which is always aware of its environment and can adapt accordingly. CR systems enable dynamic spectrum access (DSA) and thus improve the overall efficiency of spectrum usage. One of the primary cognitive tasks of a CR is to continuously monitor the frequency spectrum in order to find holes or vacant channels that can be used for secondary transmissions. A typical solution to spectrum sensing involves filtering the wideband signal with 91 a bank of narrow-band filters and monitoring each channel using classical techniques such as energy detection (ED) [71] or more recent multi-antenna based detection [72]. However, this approach requires a huge number of RF (radio frequency) components and consumes a large amount of power. An alternative is to sample the entire wideband signal at Nyquist rate and digitally monitor each channel. A primary challenge with this approach is the requirement of ADCs with very high sampling rates, which can be prohibitively expensive. Recent advances in compressive sampling (CS) have demonstrated the principle of sub-Nyquist-rate sampling and reliable signal recovery when the signals are sparse or compressible [3]. Since licensed signal transmissions are sparse in the frequency domain, CS techniques can be applied to the cognitive task of spectrum sensing. Exploiting this idea, numerous CS-based CR systems have been proposed in the past five years. Many of them use impractical sampling schemes such as those involving random Gaussian matrices and thus lack an efficient implementation [73, 74, 75, 76, 77]. Some systems use computationally intensive algorithms such as those based on, `1 norm minimization [73, 75, 76, 78], matrix rank minimization [74], matrix completion [79], PSD (power spectral density) estimation through autocorrelation [73, 76, 80] or Bayesian iterative algorithms [77]. Some methods [74, 75, 79] approximate the wideband spectrum using a spectrum vector Sf (with length equal to number of channels K) and assume that Sf = F x where F is a DFT (Discrete Fourier Transform) matrix and x (of length K) is the discretized input wideband signal. This results in a frequency spectrum with very poor resolution and high spectral leakage because of severely time-limiting the input signal. Some CS-CR systems propose cooperative sensing using fusion centers to collect the measurements [79] or signal autocorrela- 92 tions [76] and perform joint support detection through complicated algorithms. In contrast to the above, in this work, we develop a spectrum sensing algorithm (Sec. 5.3) that has the following features. The wideband signal is sampled at a subNyquist rate, according to a sampling scheme (Sec. 5.3.1) which can be efficiently implemented using low-rate ADCs. The occupied channels are identified using a simple algorithm (Sec. 5.3.2 and 5.3.3) that processes the signal samples through application of low-dimensional FFTs (Fast Fourier Transforms). The algorithm is easily implemented in a cooperative fashion, with exchange of minimal bits between onehop neighbors. Also, the algorithm can be implemented in a decentralized fashion without the need for a fusion center. Numerical simulations, in Sec. 5.4, support the theory developed in Sec. 5.3. We conclude with a discussion of future work in Sec. 5.5. 5.2 The Problem Statement The input wideband signal x(t) is assumed to be band-limited to [0, FN ] where FN is the maximum frequency in x(t). Note that FN is also the Nyquist rate. For convenience, we consider only the band of positive frequencies. The developed techniques can be easily applied to a real-valued signal band-limited to [−FN /2, FN /2]. The wideband spectrum is assumed to be divided into K non-overlapping channels, indexed by i = 0, 1, .., K − 1. Only s < K of channels are assumed to be occupied, with Is ⊂ {0, 1, .., K − 1} denoting the set containing the indices of the occupied channels. Given FN , K and s, the problem is to find the set Is . In practice, s can be assumed to be known approximately from a history of channel occupancy statistics. The problem is depicted in Fig. 5.1, where a spectrum of 120 MHz is 93 divided among K = 64 noisy channels with only s = 5 occupied or active channels (Is = {11, 21, 27, 28, 62}). The desired output of the spectrum sensing algorithm is plotted on the right, where a 1 indicates channel activity. Figure 5.1: (left) The magnitude spectrum of a wideband signal (FN = 120MHz) with s = 5 occupied channels in a total of K = 64 channels. (right) The desired output of the spectrum detection If the signal x(t) is observed for a time slot of duration tS = N/FN , the signal can be discretized as a vector x of length N , with x[n] = x(n/FN ) for n = 0, 1, .., N − 1. The N -point DFT (Discrete Fourier Transform) of x[n], denoted by X[f ] has N frequencies indexed by f = 0, 1, ..., N − 1. Assuming N = KR + 1 and ignoring the zero frequency, each channel is made up of (N − 1)/K = R discrete frequencies. Since only s channels are occupied, X is sR-sparse1 . State-of-the-art sub-linear algorithms such as [63] and [14], which reconstruct sparse vector X from random samples of x can be used to identify the occupied bands, however, the required random sampling pattern is challenging to implement in simple hardware. Also, since we only need to detect the s occupied bands and not reconstruct the entire spectrum, we use a similar but much simpler sampling scheme in our algorithm. 1A signal is called s-sparse if at most s terms are non-zero 94 5.3 The Wideband Spectrum Sensing Model A high-level block diagram of the proposed wideband spectrum sensing scheme is shown in Fig. 5.2, with explicit pseudo-code in Table 5.1. The individual blocks are explained in detail in the sections following. The input signal is sampled according to a structured random sampling pattern (detailed in Section 5.3.1). The samples are processed by R filters (Fr , r = 0, .., R−1) whose pass-bands are uniformly interleaved in the frequency domain (see Fig. 5.4). Together, the R filters form, what we call, a uniformly-interleaved filter-bank (UIFB). The structure in the sampling pattern is exploited to perform the filtering operations through low dimensional FFTs (see Section 5.3.2). This process produces random samples of the R outputs of the UIFB, denoted by xr [t], r = 0, .., R − 1, where xr [t] = (x ∗ Fr )[t]. As we will see, the UIFB divides the high dimensional input signal into R low dimensional frequency-sparse signals. The s dominant frequencies in these signals are identified by the next block and a K-length output vector b is produced. The indices of the s biggest terms in b give the set of active channels Is . The robustness of the detection algorithm is improved by taking element-wise median over J independent copies of b. The J copies are produced by the same cognitive radio or by J different cognitive radios when implemented as a collaborative sensing scheme. 5.3.1 The structured random sampling system The cognitive radio samples the vector x[n] according to the sampling pattern shown in Fig. 5.3 (top portion), where t` ∼ U [0, 1, .., N − 1] is a uniform random variable for ` = 0, 1, .., L − 1, with L = O(s logK). For each t` , the sampling pattern contains an arithmetic progression of size R (under mod-N arithmetic). As we 95 Figure 5.2: Block diagram of the spectrum sensing scheme will see in Section 5.3.2, the UIFB implementation consists of computing an R-point FFT of each arithmetic progression. The sampling scheme can be easily implemented through a multi-coset system [81], using analog-to-digital converters with a low rate of FN /K (but with cut-off frequency FN ). Figure 5.3: 5.3.2 (top) Sampling pattern of the proposed structured random sampling scheme and (bottom) random samples of UIFB outputs The Uniformly-Interleaved Filter bank (UIFB) An example of an ideal UIFB with R = 3 is shown in Fig. 5.4, where it is compared with a regular sub-band decomposition filter-bank with R = 3. The UIFB can be conceptually described as a three-step system (see Fig. 5.5). In the first step, the frequencies from different channels in x are uniformly interleaved with each other to give a new signal y. This is illustrated in Fig. 5.6 for K = 4 and R = 6. This step 96 Figure 5.4: Ideal Pass-bands of filters F0 , F1 , .. in a (left) Regular sub-band decomposition filterbank with R = 3 and (right) a uniformly-interleaved filter-bank (UIFB) with R = 3. is carried out through a time dilation t 7→ Kt mod N . In the Fourier domain, this translates to the frequency mapping f 7→ K −1 f mod N , ∀f = 0, 1, .., N − 1, where K −1 = (K − 1)R + 1 is the multiplicative inverse of K under mod-N arithmetic, i.e. KK −1 mod N = 1. The second step consists of passing y through a regular sub-band decomposition filter bank with R band-pass filters that cover the entire spectrum. If h is a low pass filter with R taps whose cutoff frequency is about π/R radians, then the sub-band decomposition filter bank in the second step, can be constructed by modulating h to different frequency bands. That is, hr [t] = ej(2r+1)πt/R h[t] for r = 0, .., R − 1. For simplicity, we use the boxcar filter with R taps, i.e. h[i] = 1 for i = 0, .., R − 1. It is possible that more sophisticated low-pass filters will sometimes yield better results. In the final step, the frequencies in each filter output are restored to their original places, by carrying out a reverse time dilation t 7→ K −1 t mod N . The entire process thus has the effect of passing the signal x through a filter bank in which the passbands of different filters are uniformly interleaved. It is important to emphasize that the algorithm actually implements the three steps of UIFB in a single shot (see Table 5.1). Mathematically, the rth output of the 97 Figure 5.5: A conceptual block diagram of the uniformly-interleaved filter bank Figure 5.6: (left) Input signal spectrum with K = 4 channels (N = 25), (right) signal spectrum after uniform frequency interleaving through mapping f 7→ 19f mod 25 which corresponds to a time dilation t 7→ 4t mod 25 UIFB xr [t] can be expressed as, xr [t] = R−1 X h[i]x[t − Ki]e j(2r+1)πi R i=0 = R−1 X x[t + Ki]e− jπi R e− j2πri R i=0 Given a time point t` , the outputs of the filter bank xr [t` ] for r = 0, .., R − 1 can be simultaneously calculated by extracting the signal samples at arithmetic progression as shown in Fig. 5.3, multiplying them by e−jπi/R , i = 0, .., R − 1, and computing an R-point FFT. Thus, the R outputs of the UIFB can be randomly sampled at t` , ` = 0, 1, .., L − 1, through computing R-point FFTs on the structured random samples of the input signal. In Prop. V.1, we prove that the desired uniform interleaving of channel frequencies 98 can be obtained as shown in Fig. 5.6. Proposition V.1. The rth output of UIFB xr [t] captures the (r +1)th frequency from every channel, for r = 0, 1, .., R − 1, assuming ideal rectangular filters in the UIFB. Proof. For r = 0, 1, .., R − 1, the rth filter of the UIFB captures (N − 1)/R = K frequencies of x[t] that get mapped to the rth filter pass-band, which is made up of the frequencies of y[t] indexed by {(rK + i + 1), i = 0, 1, .., K − 1}. Let fri denote the frequency of x[t] that gets mapped to the frequency (rK + i + 1) of y[t]. We have, fri = [(rK + i + 1)K −1 ] mod N = [(rK + i + 1)((K − 1)R + 1)] mod N = [r(K − 1)KR + rK + i((K − 1)R + 1) + (K − 1)R + 1] mod N = [r(K − 1)(KR + 1) + r + i(KR + 1) − iR + (K − 1)R + 1] mod N = [(r(K − 1) + i)(KR + 1) + ((K − 1) − i)R + r + 1] mod N = [(r(K − 1) + i)N + ((K − 1) − i)R + r + 1] mod N (putting N = KR + 1) = ((K − 1) − i)R + r + 1 Thus, the rth filter output xr [t] captures the frequencies of x[t] indexed by {1 + r, 1 + r + R, 1 + r + 2R, ..., 1 + r + (K − 1)R}, which are the (r + 1)th frequencies of all the K channels. For example, the filter output x0 [t] captures the first frequency from all the K channels, given by the indices {1, 1 + R, 1 + 2R, ..., 1 + (K − 1)R}. Thus the UIFB achieves uniformly interleaved filtering. Proposition V.2. The rth output of UIFB xr [t] captures the frequencies of x(t) that belong to the class {(r + 1) mod R}, for r = 0, 1, .., R − 1, assuming ideal rectangular 99 filters in the UIFB. Proof. From Prop. V.1, xr [t] captures the frequencies indexed by fri = ((K − 1) − i)R + r + 1 for i = 0, 1, .., K − 1. Now, fri mod R = (((K − 1) − i)R + r + 1) mod R = r + 1. Thus fri belongs to the class {(r + 1) mod R}. In other words, fri = iR + r + 1 (from Proposition V.1 with change of variables i to (K − 1) − i). This is illustrated for x0 [t] in Fig. 5.7. 5.3.3 Frequency Identification From Prop. V.1 and Prop. V.2, we see that each of the K channels contribute a single frequency to each signal xr [t]. Since only s of the K channels are assumed to be occupied, each signal xr [t] is s-sparse in frequency domain (in practice, due to the non-ideal nature of the filters in the UIFB, each xr [t] will have non-zero frequencies other than the the dominant s-frequencies captured by the ideal filter). If b(r) denotes the vector containing the coefficients of the K frequencies that are captured by xr [t], then b(r) is s-sparse. Let y(r) = [xr [t0 ], ..., xr [tL−1 ]]T be the vector that contains the L random samples of xr [t]. We can relate b(r) and y(r) with the under2π determined linear system B(r)b(r) = y(r), where B(r)`,i = ej N fri t` , ` = 0, 1, .., L − 1, i = 0, 1, .., K − 1 and fri = iR + r + 1 (from Proposition V.2). The linear system B(r)b(r) = y(r) is shown in Fig. 5.8. The s non-zero terms of each b(r) are identified by applying iterative thresholding (IT) [8] to each set of equations B(r)b(r) = y(r) (see Table 5.1). Since 100 Figure 5.7: (top) Input signal spectrum with K = 4 channels (N = 25) and R = 6 frequencies per channel, (middle) signal spectrum after uniform frequency interleaving through mapping f 7→ 19f mod 25 which corresponds to a time dilation t 7→ 4t mod 25. Also shown are the R = 6 pass-bands of the sub-band decomposition filter bank, (bottom) signal spectrum at the output of the first filter in the UIFB. b(r),r = 0, 1, .., R − 1 are jointly sparse, i.e. they have the same non-zero support, the IT can be modified and applied in a joint fashion over B(r)b(r) = y(r), ∀r. However for simplicity and to achieve parallelize-ability, we apply IT to obtain an estimate b̃(r) of each b(r) and combine the outputs at the end utilizing their joint sparsity. For a vector z, z(s) is defined as the best s-term approximation to z, which can 101 Figure 5.8: Figure showing the various terms in the linear system B(r)b(r) = y(r). be obtained by setting all the elements of z to zero except the dominant (in magnitude) s-terms. The function Φ(a) is the indicator function which is defined as 1 if a 6= 0 and zero otherwise. The algorithm gets an estimate of the non-zero terms in b(r) by performing the multiplication B(r)H y(r) and refines this estimate with each iteration. We observed that reps = 5 is enough in practice. At the end of iterations, the algorithm converts b̃(r) into a votes vector by using the indicator function Φ(.). Hence, the votes vector Φ(b̃(r)) consists of 10 s at the indices corresponding to the active channels identified from b̃(r). Since, b(r) are jointly sparse, we can obtain a P more robust estimate of the active channels by performing b = R−1 r=0 b̃(r). The set of active channels is obtained as Is = supp(b(s) ), where supp(z) is defined as the set of indices where z is non-zero. Proposition V.3. If L = O(s log K), then the frequency identification algorithm finds the correct set Is , with high probability. 102 input: {t` , ` = 0, 1, .., L − 1}, K, R, s. output: b (length K, sum of votes from each b̃(r)) UIFB: (from Sec. 5.3.1 and Sec. 5.3.2) for ` = 0, 1, .., L − 1, define zr = x((t` + rK)modN ∀r = 0, 1, .., R − 1 ),−jπr/R [x0 [t` ], .., xR−1 [t` ]] = FFT zr e , r = 0, 1, .., R − 1 Frequency identification: (from Sec. 5.3.3) for r = 0, 1, .., R − 1, b̃(r) = 0, residual e(r) = y(r) for i = 1, .., reps, b̃(r) = [B(r)H e(r) + b̃(r)](s) e(r) = y(r) − B(r)b̃(r) b̃(r) = Φ(b̃(r)) PR−1 b̃(r) b = r=0 Table 5.1: The Spectrum Sensing Algorithm Proof. If L = O(−2 slogK), then the algorithm correctly identifies the s-non zero terms of each b(r) with probability greater than 1 − O(2 ) (from a similar theorem in [16]). The failure probability O(2 ) is further reduced to O(2 )/R as b combines the R different estimates of b (namely b̃(r),r = 0, 1, ..., R − 1). Each frequency identification block has a run-time of O(sK logK). The run-time of the spectrum sensing algorithm is thus dominated by the term O(sRK logK). The entire process is illustrated for a signal with K = 4 and s = 2 in Fig. 5.9 (assuming ideal conditions, i.e. the signal sparsity is sR = 12 and each xr [t] has only s = 2 non-zero frequencies). 5.3.4 Improving robustness through median operation The robustness of the algorithm is improved further by combining J independent copies of vector b. These copies are obtained by an individual cognitive radio which observes the signal for J time slots each of duration tS = N/FN . This assumes that 103 Figure 5.9: The spectrum detection scheme illustrated for a signal with s = 2 channels occupied in a total of K = 4, for R = 6. the wide band spectrum is either stationary or varying very slowly during the J time slots. When implemented in a collaborative fashion, the J copies of b are obtained by one-hop neighbors (Assuming each CR has J one-hop neighbors). Thus Klog2 R bits are exchanged between one-hop neighbors. The number of bits that are communicated can be reduced further by employing variable length coding techniques. If b(i) denotes the ith copy of vector b, i = 0, 1, .., J − 1, then the final output b is 104 obtained as b = median(b(0) , b(1) , .., b(J−1) ) The active channel set is then obtained as Is = supp(b(s) ). The advantage of taking a median instead of mean can be demonstrated with the following simple example. Let W be a random variable with unknown mean µ and variance σ 2 . Say, we want to estimate W from N independent observations of itself with failure probability of δ. Let U = mean(W1 , W2 , .., WN ) be the estimate through mean and V = median(W1 , W2 , .., WN ) be the estimate obtained through taking median. Lets say the estimate fails if it is more than 2 standard deviations away from mean. Pr(Wi bad) = Pr(|W − µ|2 > 4σ 2 ) ≤ 1/4, by Chebyshev inequality. Now, Pr(U bad) = Pr(|U − µ|2 > 4σ 2 ) ≤ 1/4N . Equating δ = 1/4N =⇒ N = 1/4δ. Lets do the same calculations for V and see how many measurements are needed to get the same failure probability. Pr(V bad) = Pr( more than half of Wi ’s are bad ) ≤ exp (−2N (0.5 − 0.25)2 ). Equating to δ we get N = 8 ln(1/δ). For δ = 10−3 for example, U -method needs 250 measurements whereas V -method needs 55 measurements. Taking median has advantage of requiring O(ln(1/δ)) measurements versus O(1/δ) required for mean. Proposition V.4. If J =O(log 1 δ ), then all the active channels are correctly iden- tified with probability greater than 1 − δ. Proof. From Proposition V.3, all the active channels are correctly identified in each b(i) with high probability. Let this probability be p > 0.5. Then Pr(all active channels are correctly identified in b) = Pr(more than half of b(i) correctly identify all the active 105 2 2 channels) ≥ 1 − e−2J(p−0.5) (from Chernoff bound). Now, 1 − e−2J(p−0.5) ≥ 1 − δ for J =O(log( 1δ )). 5.4 Simulation Results and Discussion The input signal is generated using the following model: x[n] = X ai x0 [n]e j2π iFN n K FN + ξ[n] i∈Is where iFN K correspond to center frequencies of different occupied channels for i ∈ Is and ξ[n] is additive white Gaussian noise. x0 [n] is a signal composed of randomly chosen off-grid2 frequencies ∈ [0, FN /K] with random amplitudes and phases. The coefficients ai correspond to channel gain between the ith primary transmitter and the cognitive radio. Hence, x[n] is modeled, by taking a random multi-tone signal with appropriate frequencies, and translating it in frequency domain to different bands that are to be occupied. In the following simulations, FN = 120MHz with K = R = 64, s = 5 and ai are chosen to be comparable to each other. Note that the values of FN , etc., are chosen just as an example and are not critical to the performance of the algorithm. The probability of detection3 Pd and probability of false alarm4 Pf are used as performance metrics. 2 An off-grid frequency does not lie on the Nyquist grid of frequencies and causes spectral leakage when input signal is time-limited to a finite window 3 P = Pr( all occupied channels are correctly detected ) d 4 P = Pr( any of the vacant channels are falsely detected as occupied ) f 106 5.4.1 Varying Sub-sampling Ratio In the first experiment, the SNR5 of each occupied channel is about −2 dB. The probability of detection Pd is calculated empirically over 200 repetitions. We repeat the same experiment with J = 1, 3, 5 and 9. The plots for Pd and Pf versus sub-sampling ratio (LR/N ≈ L/K) are shown in Fig. 5.10. As can be seen from the figure, in the case of J = 1, for sub-sampling ratios greater than 0.15, Pd is higher than 0.5 (demonstrating Prop. V.3) and Pd approaches a value close to 1 at L/K = 0.5. When J is increased to 3, we see substantial improvement for all sub-sampling ratios that had Pd > 0.5 for J = 1 (demonstrating Prop. V.4). Pd quickly approaches 1 at L/K ≈ 0.33. Further improvements in Pd can be obtained by increasing J. Figure 5.10: Pd (left) and Pf (right) vs. sub-sampling ratio for J = 1, 3, 5, 9 5.4.2 Varying Input SNR In the next experiment, we compare the performance of our scheme (with L/K = 0.35, 0.3, 0.25 and J = 5) with the Nyquist rate energy detector, for various values 5 SNR(dB) of each channel is calculated as the ratio of the power in channel and the power of noise ξ[n] 107 of SNR ranging from −25dB to 5dB. In a Nyquist rate energy detector, the received signal is sampled uniformly at Nyquist rate, passed through K narrowband bandpass filters and energy of each filter output is then monitored. A primary user is detected as present, if the output energy of the corresponding filter is among the top s values. The plots are provided in Fig. 5.11. As can be seen from the figures, the performance of the proposed scheme closely follows that of a Nyquist rate energy detector at different SNR values and gives the same performance when SNR > −2dB for L/K = 0.25. This improves to SNR > −10dB for L/K = 0.35. Note that in the above experiments, for example, J = 3 copies can also be viewed as obtained by an individual cognitive radio which observes and samples the signal for 3 time slots each of duration N/FN . Figure 5.11: Pd (left) and Pf (right) vs. SNR for Nyquist-rate ED and for proposed scheme with J = 5, L/K = 0.35, 0.3, 0.25 The performance of our scheme is better than those in [80, 78, 74, 75], even though we use a more realistic setting that does not ignore spectrum leakage due to time limitation. It is not clear if we perform better or worse than [73, 76, 79], due to differences in measurement schemes and performance metrics. 108 5.4.3 Varying R (Number of Frequencies per Channel) Since the signal x(t) is observed only for a finite window of time duration tS = N/FN (with N = KR + 1), the discretized signal x exhibits spectral leakage in its DFT. The spectral leakage due to time-limiting is illustrated for a baseband signal (of band width 2W ) in Fig. 5.12. Figure 5.12: (left) DTFT of a bandlimited signal with bandwidth 2W , (right) DFT of the same signal observed in a limited time window Quantitatively, the spectral leakage can be calculated as the ratio between the energy that spilled out of the channel and the total energy of the baseband signal. Increasing the value of R, increases N and the window duration tS , thus decreasing the amount of spectral leakage. Hence the performance of the algorithm improves as R increases. Increasing R, however, also increases (linearly) the runtime of the algorithm. Hence, there is a trade-off involved in the choice of R. In Fig. 5.13, the probability of detection Pd (for J = 3) and the spectral leakage (as a fraction of total energy in an occupied channel) are both plotted on y-axis. It can be observed that the probability of detection Pd increases as spectral leakage decreases (with increas109 ing R), and eventually approaches its maximum value of 1 when R is large enough. In accordance with Fig. 5.13, a value of R around 64 seems prudent as there is no advantage in increasing the value further. Figure 5.13: 5.4.4 Probability of detection Pd versus R and Spectral leakage (expressed as a fraction of total energy in an occupied channel) versus R Simple Heuristics for Estimating s (Number of Occupied Channels) The algorithm developed in Sec. 5.3 treats s as an input parameter, assuming its knowledge from a history of channel occupancy statistics. However, this value of s, hereafter referred to as sin , may slightly be in error. Fig. 5.14 shows the performance of the algorithm when sin is in error. In this section we present simple heuristics to estimate the correct value of s. We assume that sin > s. 110 Figure 5.14: Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 3, input SN R = −2 dB, s = 5 and different values of sin . Heuristic A Let s̃ denote the estimated value of s. If b(j) denotes the output vector b obtained by CR number j (as in Sec. 5.3.4), for j = 0, 1, .., J − 1, we calculate a new vector d as follows, d = sum(b(0) , b(1) , .., b(J−1) ). s̃ is then obtained as, s̃ = argmaxi |d[i] − d[i+1] | 111 where d[i] is the ith largest element of d. In other words, the output votes vectors b (from Table 5.1) for different CRs are added to obtain d. The vector d is sorted in a descending order and the difference between consecutive elements is calculated. If s = 5, then the fifth consecutive difference |d[5] − d[6] | is expected to be the largest in magnitude. The active channel set can then be obtained as before in Sec. 5.3.4 using the estimated value s̃ in place of s. Fig. 5.15 plots the Pd and Pf versus Sub-sampling ratio for the algorithm with and without the estimation of s̃. For reference, Pd and Pf when sin = s = 5 are also plotted. As expected, when sin = 8 and without estimation, Pd improves while Pf degrades, since the algorithm tries to find 8(> s) channels that are occupied. With estimation, both Pd and Pf are brought closer to the case when sin = s. Heuristic B If b(j) denotes the output vector b obtained by CR number j (as in Sec. 5.3.4), for j = 0, 1, .., J − 1, we calculate a new vector d as follows, J−1 X (j) d= φ b(sin ) j=0 where z(s) is obtained by setting all the elements of z to zero except the dominant (in magnitude) s-terms and φ(z) = 1 if z > 0, φ(z) = 0 if z = 0. For a chosen threshold λ ∈ [0, J], s̃ is then obtained as, s̃ = K−1 X φ(di ≥ λ) i=0 where φ(true) = 1 and φ(false) = 0. 112 Figure 5.15: Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 5, input SN R = −2 dB, s = 5, sin = 8, with and without the estimation of s̃ using Heuristic A. In other words, in the first step, each cognitive radio j votes for all the sin occupied channels using its output vector b(j) . In the second step, s̃ is obtained as the number of channels that received more than λ votes. The net s̃ is chosen to be the maximum among this value and the one estimated by Heuristic A. Once s is estimated, the final output vector is determined by the median operation described in Sec. 5.3.4. Fig. 5.16 plots the Pd and Pf versus Sub-sampling ratio for the algorithm with and 113 without the estimation of s̃. The threshold is chosen as λ = dJ/2e. For reference, Pd and Pf when sin = s = 5 are also plotted. Heuristic B performs better in terms of Pf compared to Heuristic A. Figure 5.16: Pd (top) and Pf (bottom) vs. Sub-sampling ratio for proposed scheme with J = 5, input SN R = −2 dB, s = 5, sin = 8, with and without the estimation of s̃ using Heuristic B. 114 5.5 Conclusion and Future work We proposed a novel spectrum sensing scheme for wideband CRs to detect spectrum opportunities, assuming low spectrum utilization. The scheme uses a multicoset type sampling to efficiently sample the signal at a sub-Nyquist rate close to the channel occupancy. We divided the input signal into several low-dimensional frequency-sparse signals using the newly developed concept of uniformly-interleaved filter bank (UIFB). The UIFB is implemented by “smart” processing of signal samples through low-dimensional FFTs. We solved the frequency identification problem through parallelize-able iterative thresholding techniques. The robustness of the scheme is improved by observing the signal for a longer duration or by collaborating with neighboring one-hop CRs, with minimal pair-wise communication. The algorithm can also be easily extended to estimate the number of occupied channels. Reducing the run-time of frequency identification to a sub-linear time is a future work of interest. Studying the performance under different types of fading channels is another future research direction. 115 BIBLIOGRAPHY 116 BIBLIOGRAPHY [1] S. Naraghi, M. Courcy, and M.P. Flynn, “A 9b 14µW 0.06mm2 PPM ADC in 90nm digital CMOS,” IEEE International SolidState Circuits Conference, vol. 54, pp. 168–169, 2009. [2] D.L. Donoho, “Compressed sensing,” IEEE Trans. Info. Theory, vol. 52(4), pp. 1289–1306, Sep 2006. [3] E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Transactions on Information Theory, vol. 52(2), pp. 489–509, February 2006. [4] M. Rudelson and R. Vershynin, “Sparse reconstruction by convex relaxation: Fourier and gaussian measurements,” 40th Annual Conference on Information Sciences and Systems (CISS), 2006. [5] A. E. Litvak, A. Pajor, M. Rudelson, and N. Tomczak-Jaegermann, “Smallest singular value of random matrices and geometry of random polytopes,” Advances in Mathematics, vol. 195(2), pp. 491–523, 2005. [6] J. A. Tropp and A. C. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Transactions on Information Theory, vol. 53(12), pp. 4655–4666, December 2007. [7] D. Needell and J.A. Tropp, “Cosamp: Iterative signal recovery from incomplete and inaccurate samples,” Applied and Computational Harmonic Analysis, vol. 26, pp. 301–321, April 2008. [8] T. Blumensath and M.E. Davis, “Iterative thresholding for sparse approximations,” Journal of Fourier Analysis and Applications, vol. 14, pp. 629–654, September 2008. [9] E. Candes and M. Wakin, “An introduction to compressive sampling,” IEEE Signal Processing Magazine, vol. 25(2), pp. 21–30, March 2008. [10] R. A. DeVore, “Deterministic constructions of compressed sensing matrices,” J. Complex., vol. 23(4), pp. 918–925, Aug 2007. [11] A.C. Gilbert, S. Muthukrishnan, and M.J. Strauss, “Improved time bounds for nearoptimal sparse fourier representation via sampling,” In Proceedings of SPIE Wavelets XI,San Diego,CA, 2005. [12] M. Iwen, “A deterministic sub-linear time sparse fourier algorithm via non-adaptive compressed sensing methods,” In Proceedings of ACM-SIAM Symposium on Discrete Algorithms (SODA), 2008. [13] P.K. Yenduri and A.C Gilbert, “Continuous Fast Fourier Sampling,” In Proceedings of Sampling Theory and Applications (SAMPTA), Marseille, France, 2009. [14] H. Hassanieh, P. Indyk, D. Katabi, and E. Price, “Simple and practical algorithm for sparse fourier transform,” In Proceedings of ACM-SIAM Symposium on Discrete Algorithms (SODA), 2012. 117 [15] S. Kunis and H. Rauhut, “Random sampling of sparse trigonometric polynomials, ii. orthogonal matching pursuit versus basis pursuit,” Foundations of Computational Mathematics, vol. 8(6), pp. 737–763, November 2008. [16] P.K. Yenduri, A.C. Gilbert, M.P. Flynn, and S. Naraghi, “Rand PPM: A low power compressive sampling analog to digital converter,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5980–5983, May 2011. [17] P.K. Yenduri, A.C. Gilbert, and J. Zhang, “Model of a sparse encoding neuron,” Twenty First Annual Computational Neuroscience Meeting (CNS), Jul. 2012. [18] P.K. Yenduri, A.C. Gilbert, and J. Zhang, “Integrate-and-fire neuron modeled as a low-rate sparse time-encoding device,” Proceedings of Third International Conference on Intelligent Control and Information Processing (ICICIP), Jul. 2012. [19] P.K. Yenduri and A.C. Gilbert, “Compressive, collaborative spectrum sensing for wideband cognitive radios,” The Ninth International Symposium on Wireless Communication Systems (ISWCS), Aug. 2012. [20] P.K. Yenduri, A. Rocca, A.S. Rao, S. Naraghi, A.C Gilbert, and M.P. Flynn, “A low power compressive sampling time-based analog to digital converter,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), Special Issue on Circuits, Systems and Algorithms for Compressive Sensing, Oct. 2012. [21] P.K. Yenduri and A.C. Gilbert, “CFFS: A sub-Nyquist sub-linear time sliding window algorithm,” In review, IEEE Signal Processing Letters, Oct. 2012. [22] J.A. Michaelsen, J.E. Ramstad, D.T. Wisland, and O. Sorasen, Low-power Sensor Interfacing and MEMS for Wireless Sensor Networks, InTech, 1 edition, 2011. [23] F. Shahrokhi, K. Abdelhalim, D. Serletis, P.L. Carlen, and R. Genov, “The 128-channel fully differential digital integrated neural recording and stimulation interface,” IEEE Transactions on Biomedical Circuits and Systems, vol. 4(3), pp. 149–161, February 2010. [24] E. Delagnes, D. Breton, F. Lugiez, and R. Rahmanifard, “A low power multi-channel single ramp ADC with up to 3.2 ghz virtual clock,” IEEE Transactions On Nuclear Science, vol. 54(5), pp. 1735–1742, October 2007. [25] T. Fusayasu, “A fast integrating ADC using precise time-to-digital conversion,” IEEE Nuclear Science Symposium Conference Record, vol. 1, pp. 302–304, 2007. [26] A.H. Reeves, Electrical Signaling System, February 1942. [27] J. Mark and T. Todd, “A non-uniform sampling approach to data compression,” IEEE transactions on Communications, vol. 29(1), pp. 24–32, January 1981. [28] M.Z. Straayer and M.H. Perrott, “A 10-bit 20MHz 38mW 950MHz CT Sigma Delta ADC with a 5-bit noise-shaping VCO-based quantizer and DEM circuit in 0.13µ CMOS,” VLSI Symp. Dig. Tech. Papers, pp. 246–247, June 2007. [29] A.A. Lazar and L.T. Toth, “Perfect recovery and sensitivity analysis of time encoded bandlimited signals,” IEEE Transactions on Circuits and Systems-I:Regular Papers, vol. 51(10), pp. 2060–2073, October 2004. [30] Aurel A. Lazar and Eftychios A. Pnevmatikakis, “Reconstruction of sensory stimuli encoded with integrate-and-fire neurons with random thresholds,” EURASIP Journal on Advances in Signal Processing, vol. 2009, 2009, Special Issue on Statistical Signal Processing in Neuroscience. 118 [31] M. Kurchuk and Y. Tsividis, “Signal-Dependent Variable-Resolution clockless A/D conversion with application to CT-DSP,” IEEE Trans. Circuits and Systems, vol. 57(5), pp. 982–991, May 2010. [32] B. Schell and Y. Tsividis, “A continuous-time adc/dsp/dac system with no clock and with activity-dependent power dissipation,” IEEE Journal of Solid-State Circuits, vol. 43(11), pp. 2742–2481, Nov. 2008. [33] Y. Li, “A 0.5v signal-specific continuous-time level-crossing adc with charge sharing,” IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 381–384, Nov. 2011. [34] B. Murmann, “A/D converter trends: Power dissipation, scaling and digitally assisted architectures,” IEEE Custom Integrated Circuits Conference (CICC), pp. 105–112, 2008. [35] J.N. Laska, S. Kirolos, M.F. Duarte, T.S. Ragheb, R.G. Baraniuk, and Y. Massoud, “Theory and implementation of an analog-to-information converter using random demodulation,” IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1959–1962, 2009. [36] C. Luo and J.H. McClellan, “Compressive sampling with a successive approximation adc architecture,” International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3920–3923, 2011. [37] S.R. Becker, “Practical compressed sensing: modern data acquisition and signal processing,” in Dissertation (Ph.D.). California Institute of Technology, 2011. [38] M. Mishali, Y.C. Eldar, and A.J. Elron, “Xampling: Signal acquisition and processing in union of subspaces,” IEEE Transactions on Signal Processing, vol. 59(10), pp. 4719–4734, October 2011. [39] F. Chen, A.P. Chandrakasan, and V. Stojanovic, “A signal-agnostic compressed sensing acquisition system for wireless and implantable sensors,” Custom Integrated Circuits Conference (CICC), pp. 1–4, 2010. [40] M. Fornasier and H. Rauhut, Compressive Sensing, Springer, 2011. [41] S.G. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Trans. Signal Processing, vol. 41(12), pp. 3397–3415, 1993. [42] S. Szczepnski P. Dudek and J.V. Hatfield, “A high-resolution CMOS time-to-digital converter utilizing a vernier delay line,” IEEE Journal on Solid-State Circuits, vol. 35(2), pp. 240–247, February 2000. [43] F. Marvasti, Nonuniform Sampling Theory and Practice, Kluwer Academic Publisher, 1990. [44] F. Marvasti, M. Analoui, and M. Gamshadzahi, “Recovery of signals from nonuniform samples using iterative methods,” IEEE Transactions on Signal Processing, vol. 39(4), pp. 872–878, April 1991. [45] R.G. Baraniuk and P. Steeghs, “Compressive radar imaging,” IEEE Radar Conf., pp. 128–133, Apr. 2007. [46] F.J. Herrmann, D. Wang, G. Hennenfent, and P. Moghaddam, “Curvelet-based seismic data processing: A multiscale and nonlinear approach,” Geophysics, vol. 73(1), pp. A1–A5, Feb. 2008. [47] J. Berent, P. Dragotti, and T. Blu, “Sampling piecewise sinusoidal signals with Finite Rate of Innovation methods,” IEEE Trans. Sig. Proc., vol. 58(2), pp. 613–625, February 2010. [48] A. Bjorck, Numerical Methods for Least Squares Problems, Philadelphia: SIAM, 17 edition, 1996. 119 [49] G. Steidl, “A note on fast Fourier transforms for nonequispaced grids,” Advances in Computational Mathematics, vol. 9, pp. 337–352, November 1998. [50] M. Charikar, K. Chen, and M. Farach-colton, “Finding frequent items in data streams,” Theoretical Computer Science, vol. 312, pp. 3–15, January 2004. [51] A.A. Lazar and L.T. Toth, “Perfect recovery and sensitivity analysis of time encoded bandlimited signals,” IEEE Trans. on Circuits and Systems-I: Regular Papers, vol. 51, pp. 2060–2073, 2004. [52] A.A. Lazar, “Time encoding with an integrate-and-fire neuron with a refractory period,” Neurocomputing, vol. 58-60, pp. 53–58, Jun. 2004. [53] A.A. Lazar, E.A. Pnevmatikakis, and Zhou Y., “Encoding natural scenes with neural circuits with random thresholds,” Vision Research Special Issue on Mathematical Models of Visual Coding, vol. 50(22), pp. 2200–2212, 2010. [54] E.D. Adrian, The Basis of Sensation: The Action of the Sense Organs, Christophers (London), 1928. [55] D. Attwell and Laughlin S.B., “An energy budget for signaling in the grey matter of the brain,” Journal of Cerebral Blood Flow and Metabolism, vol. 21, pp. 1133–1145, 2001. [56] V. Balasubramanian and M. J. Berry, “A test of metabolically efficient coding in the retina,” Network: Computation in Neural Systems, vol. 13(4), pp. 531–552, 2002. [57] A. Manwani P. N. Steinmetz and C. Koch, “Variability and coding efficiency of noisy neural spike encoders,” BioSystems, vol. 62(1-3), pp. 87–97, 2001. [58] G. Gestri, H.A.K. Mastebroek, and W. H. Zaagman, “Stochastic constancy, variability and adaptation of spike generation: performance of a giant neuron in the visual system of the fly,” Biological Cybernetics, vol. 38(1), pp. 31–40, 1980. [59] I.F. Akyildiz, W.Y. Lee, M.C. Vuran, and S. Mohanty, “Next generation dynamic spectrum access cognitive radio wireless networks: A survey,” Computer Networks Journal (Elsevier), vol. 50, pp. 2127–2159, Sep 2006. [60] Simon Haykin, Communication systems, John Wiley and Sons, 4 edition, 2005. [61] A.C. Gilbert, M.J. Strauss, J.A. Tropp, and R. Vershynin, “Algorithmic linear dimension reduction in the l1 norm for sparse vectors,” Allerton Conference, 2006. [62] G. Cormode and S. Muthukrishnan, “Combinatorial algorithms for compressed sensing,” IEEE Int. Conf. on Information Sciences Systems, pp. 230–294, April 2006. [63] A.C. Gilbert, M.J.Strauss, and J. A. Tropp, “A tutorial on fast fourier sampling,” IEEE Sig.Proc.Mag., vol. 25(2), pp. 57–66, 2008. [64] M. Iwen, A.C. Gilbert, and M.J. Strauss, “Empirical evaluation of a sub-linear time sparse DFT algorithm,” Commun.Math.Sci, vol. 5(4), pp. 981–998, 2007. [65] G.K. Smith and D.M.Hawkins, “Robust frequency estimation using elemental sets,” J.Comput.Graph.Stat, vol. 9(1), pp. 196–214, 2000. [66] G. Harikumar and Y. Bresler, “FIR perfect signal reconstruction from multiple convolutions: minimum deconvolver orders,” IEEE Trans.Signal Processing, vol. 46(1), pp. 215–218, 1998. [67] O. Mustapha, M. Khalil, G. Hoblos, H. Chafoukand, and D. Lefebvre, “Abrupt change detection algorithm: from theory to implementation,” Colloque Evaluation des performances et matrise desrisques technologiques pour les systmes industriels et nergtiques, pp. 28–28, May 2009. 120 [68] S. Rezk, C. Join, S.E. Asmi, M. Dogui, and M.H. Bedoui, “Frequency change-point detection in physiological signals : an algebraic approach,” IJ-STA, vol. 2(1), pp. 456–468, 2008. [69] J. Berent, P. Dragotti, and T. Blu, “Sampling piecewise sinusoidal signals with finite rate of innovation methods,” IEEE Trans. Sig. Proc., vol. 58(2), pp. 613–625, Feb 2010. [70] J. Mitola and Jr. Maguire, “Cognitive radio: Making software radios more personal,” IEEE Pers. Commun., vol. 6, pp. 13–18, 1999. [71] R. Tandra and A. Sahai, “SNR walls for signal detection,” IEEE J. Sel. Top. Signal Process., vol. 2, pp. 4–17, 2008. [72] R. Lopez-Valcarce, G. Vazquez-Vilar, and M. Alvarez-Diaz, “Multiantenna detection of multicarrier primary signals exploiting spectral a priori information,” CROWNCOM, pp. 1–6, Jun 2009. [73] Y.L. Polo, Ying Wang, A. Pandharipande, and G. Leus, “Compressive wide-band spectrum sensing,” IEEE ICASSP, pp. 2337 – 2340, 2009. [74] Y. Wang, Z. Tian, and C. Feng, “Cooperative spectrum sensing based on matrix rank minimization,” IEEE ICASSP, pp. 3000 – 3003, 2011. [75] Q. Ling and Z. Tian, “Decentralized support detection of multiple measurement vectors with joint sparsity,” IEEE ICASSP, pp. 2996 – 2999, May 2011. [76] Y. Wang, A. Pandharipande, Y.L. Polo, and G. Leus, “Distributed compressive wide-band spectrum sensing,” Info. Th. App. Workshop, pp. 178 – 183, May 2009. [77] G. Vazquez-Vilar, R. Lopez-Valcarce, C. Mosquera, and N. Gonzalez-Prelcic, “Wideband spectral estimation from compressed measurements exploiting spectral a priori information in cognitive radio systems,” IEEE ICASSP, pp. 2958 – 2961, March 2010. [78] Z. Tian, “Compressed wideband sensing in cooperative cognitive radio networks,” IEEE GLOBECOM, pp. 1 – 5, Dec 2008. [79] S. Corroy, A. Bollig, and R. Mathar, “Distributed sensing of a slowly time-varying sparse spectrum using matrix completion,” ISWCS, pp. 296 – 300, Nov 2011. [80] M. Rashidi, K. Haghighi, A. Panahi, and M. Viberg, “A NLLS based sub-Nyquist rate spectrum sensing for wideband cognitive radio,” IEEE DySPAN, pp. 545 – 551, May 2011. [81] R. Venkataramani and Y. Bresler, “Perfect reconstruction formulas and bounds on aliasing error in sub-Nyquist nonuniform sampling of multiband signals,” IEEE Trans. Info. Th., vol. 46(6), pp. 2173–2183, Sep 2000. 121

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement