Black-Box Modelling of Integrated Circuits for Electromagnetic Compatibility Simulations

Black-Box Modelling of Integrated Circuits for Electromagnetic Compatibility Simulations
KVALIFIKACIJSKI DOKTORSKI ISPIT
1
Black-Box Modelling of Integrated Circuits for
Electromagnetic Compatibility Simulations
Marko Magerl
University of Zagreb Faculty of Electrical Engineering and Computing
Unska 3, Zagreb, Croatia
Email: [email protected]
Index Terms—black-box modelling, artificial neural networks,
echo state networks, electromagnetic compatibility, direct power
injection, circuit simulations.
output weights
(trained)
input weights
(trained)
u(n)
y(n)
...
Abstract—This paper presents an overview of the
state-of-the-art in black-box modelling with focus on modelling
integrated circuits using artificial neural networks. The trade-offs
of different model architectures are discussed, including the
model building time, training algorithm complexity, and
model execution time in circuit simulators. An interchangeable
black-box model of a bandgap reference block for direct
power injection simulations, based on the echo state network
architecture, is presented and discussed. Guidelines for the
future work in the research of the topic are given.
output neurons
input neurons
internal neurons
Fig. 1: Generic diagram of a single-layer feedforward neural
network (SLFN).
I. I NTRODUCTION
T
HE growing complexity of integrated circuits (IC) has
made the black-box modelling approach very appealing
to the IC industry for reducing the complexity of the circuit
simulation models. The electromagnetic compatibility (EMC)
simulations are becoming increasingly important for IC manufacturers as they can reduce the number of prototypes in the
development cycle of a product.
The idea behind the black-box, or behavioural, approach is
to model the behaviour of the modelled system without using
the information about its inner structure [1]. The advantages
of this approach are the model order reduction (MOR), which
can enable the reduction of the circuit simulation time, and the
intellectual property (IP) protection, which makes the models
free for the distribution to third parties.
In this context, a research project is established that investigates the applicability of the black-box approach to modelling
integrated circuits for EMC applications. The primary focus of
this research project is to use artificial neural networks (ANN)
to model electronic circuits in the time-domain. The goal
of this paper is to present the state-of-the-art in black-box
modelling, as well as to present initial results of the research
project.
This paper is organised as follows. An overview of the
black-box modelling theory is given in Section II. Section III
presents the state-of-the-art in black-box modelling of circuits
and for other applications. The initial results of this research
are presented in Section IV, and the guidelines for future work
are given in Section V. The paper is concluded in Section VI.
II. B LACK -B OX M ODELLING T HEORY
Artificial neural networks (ANN) are networks of interconnected units – neurons. Each neuron is associated with
a time-domain signal, the neuron activation which is defined
by the interaction between the neuron and the neighbouring
connected neurons. Different neural network architectures are
defined by the configuration of the neurons in the network and
by the weights of the connections between them.
Two basic types of ANN exist: feedforward and recurrent
neural networks. The feedforward neural networks, shown
schematically in Fig. 1, do not contain any circular paths
between the neurons. It is proven that the single-layer feedforward neural network (SLFN), shown in Fig. 1, has the
potential to approximate any continuous function [2], [3]. This
is referred to as the universal approximation property.
The feedforward architectures do not retain any information
about previous input values, therefore, they cannot be used to
model systems with memory, such as most electronic circuits.
In order to incorporate the temporal component in the model,
feedback connections between neurons are added, resulting
in recurrent neural networks (RNN). In [4] it is shown that
the internal dynamics of an RNN with random fixed weights
resemble a Markovian process if the the input connection
weights have small values. This observation forms the basis
for the echo state network (ESN) approach, as shown in [5].
The echo state network is discussed in Section II-D.
KVALIFIKACIJSKI DOKTORSKI ISPIT
2
B. ANN training algorithms
Fig. 2: Flow chart for building a generic artificial neural
network (ANN) model.
A. Model building
The flow chart for building a generic ANN model is shown
in Fig. 2. The hyper-parameters are parameters specific to a
particular ANN architecture, e.g. the number of neurons, the
weight distributions, the activation functions, the leaking rate,
the number and value of delays of the input signal, the sparsity
of the neuron connections. Adjusting the hyper-parameters is
usually done in a trial-and-error process.
Generating the ANN corresponds to initializing the data
structures that define the architecture of the particular ANN.
Within this research project, the ANN models are coded in
R
Python and MATLAB
.
In order to model a particular function or system, the ANN
is trained using data from the training dataset – a collection
of input-output values of the modelled system, obtained either
from simulations, e.g. SPICE simulations of an electronic circuit, or from measurements. An overview of the ANN training
algorithms is presented in Section II-B.
In order to test its generalisation properties, the trained
model is verified using a dataset that does not include the
data points used in training. If the model performance is not
satisfactory, the hyper-parameters are re-adjusted and the flow
is reiterated. Once a satisfactory model is obtained, the ANN
is implemented in a hardware description language (HDL) in
order to be usable in a circuit simulator. This topic is discussed
in Section II-C.
Once a particular architecture of the ANN is chosen,
the ANN is trained by adjusting the weights of the connections
according to the learning data (training dataset) based on a
training algorithm.
The “classical” training algorithms are iterative methods
based on gradient descent on an error function defined between
the desired and the actual model output. Commonly used
examples include the Bayesian regularisation backpropagation
algorithm [6], and the Levenberg-Marquardt algorithm [7].
These algorithms are readily available in software packages
for neural networks, e.g. the Neural Network ToolboxTM in
R
MATLAB
.
An overview of the gradient descent algorithms is available in [8]. These algorithms have inherent limitations for
recurrent neural networks. E.g. such a problem is bifurcations,
a phenomenon where a small change in a parameter of a
nonlinear system results in the fundamental change of the
nature of its state trajectories [9]. Bifurcations that occur
during an iterative learning process can significantly slow
down the learning process, or completely prevent the algorithm
from reaching the optimal solution [10]. Further problems
include the information latching problem, caused by the error
function vanishing in time, which limits the memory ability
of the RNN [4]; the convergence can be very slow for large
networks [10]; the algorithms may get stuck at local optimum
points that are not optimal if the optimization problem is not
convex [11].
The above training methods are offline methods, because
the complete training dataset is available during learning.
Online learning methods adjust the connection weights as
the training data is being applied to the network, and also
during the execution of the model. The advantage of this
approach is the ability of the model to adaptively respond to
disturbances during the model execution. Online algorithms
include the least mean square (LMS) and the recursive least
squares (RLS) algorithms. In [12] the RLS algorithm with the
adaptive forgetting factor is used to adjust the weights of an
echo state network.
In [13] the training algorithm is taken a step further, by
using an evolutionary algorithm that adjusts not only the
connection weights of an echo state network (ESN), but it
also chooses the optimal hyper-parameters. Similarly, in [14]
the particle swarm optimisation (PSO) evolutionary algorithm
is used to adjust the output layer weights of an ESN in an
online fashion.
C. Model implementation in circuit simulators
A number of hardware description languages (HDL)
supports analog and/or mixed signals: Verilog A,
VHDL-AMS, MAST [15]. In the context of this project, the
R
R
Cadence
Spectre
circuit simulator is used to simulate
black-box models coded in Verilog A.
An example of implementing a 3-layer feedforward neural
network in Verilog A is given in [16]. The presented methodology can be extended to different types of neural networks,
KVALIFIKACIJSKI DOKTORSKI ISPIT
D. Echo state networks
This section presents an overview of the echo state network (ESN), and Section IV presents the initial results of this
research project based on using the ESN.
The echo state network, first proposed in [10], is a recurrent
neural network that has fixed input connections and fixed
internal connections, both with randomly generated weights.
A generic diagram of the echo state network is shown
in Fig. 3. The activations of the hidden neurons represent
“echoes” of the input sequence history due to the echo state
property (ESP) of the ESN. The ESP is conceptually based
on the architectural bias of the RNN towards a Markovian
process [5]. The existence of the ESP can be mathematically
guaranteed by imposing certain conditions on the internal
weight matrix of the ESN, as it is discussed in [17]. The
rich dynamics in the hidden layer of the ESN, available due
to the ESP, enable constructing the output of the ESN using
linear regression only on the output connection weights. This
property makes the ESN training algorithm extremely fast and
robust compared to the RNN training algorithms presented in
Section II-B. An additional benefit of having a fast training
algorithm is a significantly reduced time required for tuning
the hyper-parameters according to the flow chart in Fig. 2.
Due to these favourable properties, the echo state network is
chosen as the starting point for this research project.
The numerical properties of the trained ESN are determined
by the linear regression method used in the ESN training
process. Most researchers use ridge regression, also known
as regression with Tikhonov regularisation, e.g. [10], [18], or
the simple pseudo-inverse solution, e.g. [19]. These methods
are based on minimising the L2-norm of the error vector.
An alternative method is the LASSO regression [20], which
minimises the sum of the L2-norm of the error vector and
the L1-norm of the output weights vector. It is shown in [20]
that the LASSO method forces most output weights to zero,
internal weights
input weights
output feedback
weights
output weights
(trained)
Ub
y(n)
u(n)
...
e.g. the ESN. Furthermore, the HDL code generation can be
automated using a scripting language, e.g. Python.
The two basic figures of merit for black-box models are
the building time and the execution time. In the context of
using the black-box models in circuit simulators, the model
execution time is critical because the primary goal of using
the black-box approach is the reduction of the simulation time.
It is empirically observed that the number of neurons has the
main influence on the execution time, because the neurons
are implemented in a HDL as interconnected electrical nodes.
Therefore the number of neurons determines the number of
simultaneous nonlinear equations the simulator has to solve.
Another consideration for the execution time is the smoothness
of the signals produced by the model because discontinuities
or sudden jumps in the signals cause the simulator to reduce
the timestep, increasing the simulation time. The execution
time can also be significantly improved by pre-compiling
the HDL model in a low-level language such as C. This
conversion can be done automatically, e.g. using a feature of
R
R
the Cadence
Spectre
circuit simulator, called the compiled
model interface (CMI).
3
output neurons
input neurons
x(n)
internal neurons
Fig. 3: Generic diagram of the echo state network (ESN).
generating a sparse solution. Finally, the elastic-net regression
is a method that combines the above two approaches and their
favourable properties [20].
Adding a small noise term to the training data obtained
from simulations is shown to improve the stability and generalisation properties of the trained ESN according to [10].
Although without a formal proof, the reasoning behind this
idea is that adding noise to the numerical values of the training
dataset reduces the risk of the ESN to overfit the training data,
degrading its generalisation properties.
The echo state network quickly became popular in the
research community due to its conceptual simplicity, as well as
its fast training algorithm and good generalisation properties.
However, it is shown that many theoretical results are not well
understood by the users of the ESN. For example, the original
work by H. Jaeger [10] gives an “almost sufficient” condition
for the echo state property that involves globally rescaling the
internal weight matrix of the ESN such that its spectral radius
is less than one. Many research papers, e.g. [19], [21], [22],
readily use this condition citing it as the sufficient condition for
the existence of the ESP. In [17] Jaeger shows that examples
can be found where the ESP does not hold even if the spectral
radius of the “almost” sufficient condition is satisfied. In that
work, as well as in [23], alternative methods for constructing
the internal weight matrix are proposed and evaluated.
In [18] a better intuitive understanding of the ESN is
attempted by constructing several deterministic ESN configurations and testing them using the standard benchmark tests.
In [24] the average state entropy (ASE) is introduced as a
quantification of the ESN reservoir “richness”, and the ESN
architecture is interpreted from the standpoint of signal processing of basis functions. In [25] the theoretical properties of
the ESN are further investigated. The reason why the “almost
sufficient” condition for the existence of the ESP often holds
is elaborated mathematically.
The ESN has two types of feedback connections: the internal feedback connections in the hidden layer, and the feedback
connections from the output layer to the hidden layer. While
the internal feedback connections are a fundamental part of
the ESN, the output feedback connections are optional [10].
The mathematical conditions that guarantee the echo state
property given in [17] apply only in the case the output
feedback connections are not used. In [26] the stability of
KVALIFIKACIJSKI DOKTORSKI ISPIT
the ESN with output feedback connections is investigated. The
authors propose regularising the output layer and the hidden
layer using Tikhonov regularisation. In [27] an ESN with
output feedback connections is replaced by an ESN without
feedback connections, however the training process is modified
to include incremental changes in the internal weight matrix
that reflect the influence of the output feedback connections
of the original ESN. In this way, the stability of the ESN
is improved by removing the output feedback connections,
while retaining the favourable properties of using the output
feedback connections.
In [28] the experimental results of [10] are re-evaluated, and
a critical assessment is made. The stochastic nature of the ESN
generation process is shown to influence the performance of
certain ESN instances that have the same hyper-parameters.
In [29] the influence of the spectral radius on the performance of an ESN with 50 neurons is evaluated experimentally,
showing that using the spectral radius close to 0.8 gives
optimal results.
In [30] the short term memory of the ESN and its continuous-time counterpart, the liquid state machine (LSM),
is assessed by introducing a simplified ESN (SESN) and
comparing its memory performance to the ESN and the LSM.
III. S TATE OF THE A RT
Coming from the area of computer science, artificial neural
networks are traditionally evaluated using standard benchmark
test cases, such as predicting discrete time-series: e.g. a
2nd order dynamical system, such as in [8], the NARMA
series [23], or the Mackey-Glass chaotic series [10], [24], [28].
The reservoir sizes used for these benchmarks are in the order
of several hundred neurons (compare [10]). This paper will
focus on examples from literature in which neural networks are
applied to practical engineering problems. As it is discussed
in Section II-C, keeping a low number of neurons is key to
achieving the simulation speed-up in the circuit simulator.
In [19] neural networks are used for the model predictive
control (MPC) of an unknown nonlinear system. The ESN
architecture is used for system identification, and the control
problem is formulated as a quadratic programming (QP)
problem that is solved using a single-layer neural network,
referred to as the simplified dual neural network (SDN).
In [21] and [31] the ESN is used in a harmonic extraction
block of a shunt active power filter (SAPF). The SAPF eliminates the higher harmonics from the output current caused by
nonlinear loads in power systems. The ESN is used to extract
the fundamental component of the load current. Although 100
neurons are used in [31] to obtain good performance, it
is shown in [21] that an ESN with only 30 neurons is
sufficient. The achieved performance is shown to be superior
to a feedforward neural network, referred to as the multilayer
perceptron (MLP).
In the electronic circuit modelling applications, [32]
presents the model of an output buffer of a digital IC for
signal integrity simulations. A combination of feedforward and
recurrent neural networks is used. Single-layer feedforward
neural networks (SLFN) with 4 neurons are used to model the
4
the I-V characteristics of the buffer transistors, and the total
buffer output current is modelled by weighting the currents of
the output transistors, where the weights are implemented as
recurrent neural networks with 10 neurons each. This model
overcomes the limitation of using the I/O buffer information
specification (IBIS) [33] that cannot properly model analog
nonlinear behaviour, such as ringing and attenuation. In [34]
a similar test case is presented using an ESN with 30 neurons.
The stability of the model is shown by plotting the locus of
the linearised model eigenvalues.
In [35] a 3-layer feedforward neural network is used to
model the response of a digital IC to a radiofrequency (RF)
disturbance on its supply pin. The input of the model is the RF
disturbance signal and its derivative and, in order to have
a constant number of input samples per period for varying
frequency, the sampling rate of the input data is adjusted to
the input frequency of the signal. The model is implemented
in the circuit simulator as a mathematical expression.
In [36] a 2-layer feedforward neural network is used to
model a voltage regulator for direct power injection (DPI)
simulations. In order to enable the model to interact with the
previous stage, the input impedance of the circuit is modelled
as well. The training dataset is obtained by sampling the
RF frequency–power design space using the LOLA-Voronoi
adaptive sampling algorithm [37], which enables putting more
focus on the more nonlinear regions of the design space.
IV. R ESULTS
A. EMC modelling test case
In this section, the initial results of this research project are
presented. A circuit model for direct power injection (DPI)
simulations is built and evaluated.
DPI is an electromagnetic compatibility precompliance test
for measuring the immunity of integrated circuits to electromagnetic interference (EMI). It is defined in the IEC-62132-4
standard [38]. A radiofrequency (RF) disturbance signal of
increasing forward power is injected into the IC pin under
test until the functionality of the circuit fails according to a
predefined criterion. The frequency of the RF disturbance is
swept between 150 kHz and 1 GHz.
A testbench for DPI simulations is shown in Fig. 4a.
A bias-tee circuit is used to superpose an RF disturbance
signal vRF on a DC voltage VDC , in this case on the power
supply of the device under test (DUT).
Being able to simulate the performance of an IC on the DPI
test before chip fabrication is extremely useful for IC manufacturers, as it reduces the development cost of the product
and shortens the time-to-market. In [39] an electrical model of
the measurement setup for DPI is presented for use in SPICE
simulations. Included in the model are the IC package, the
injection probe, the printed circuit board (PCB), the capacitors
for decoupling and the IC itself. However the time-domain
simulations in [39] are shown to be very slow due to the
complex netlist of the test IC. A similar pattern is observed
throughout the IC industry: the DPI simulations are inherently
slow, especially if the extracted parasitics are included in the
netlist, which is often required in order to observe realistic EMC problems in the IC.
KVALIFIKACIJSKI DOKTORSKI ISPIT
5
iin
430 µH
VDC
iout
v(vin )
+
6.8 nF
vin
VDD
DUT VBG
GN D
−
VDD
GN D
VDIGREG
BLKS
VOU T
VDD
OB
GN D
VOU T
VIN
IIN,BG VIN,BG
VBG
−
Fig. 5: Black-box model architecture.
(a) The top-level testbench.
VOUT,BG
i(vin )
vout
1 MΩ
0.001 Ω
VDD
BG
GN D
−
+
vRF
+
VBG
(b) The schematic of the DUT.
Fig. 4: The schematics used for DPI simulations.
In this research project, an interchangeable black-box model
of a bandgap reference block for direct power injection (DPI)
simulations is built using the echo state network (ESN) approach. The goal is to improve the simulation-speed of the DPI
simulations in the circuit simulator, as well as to hide the
structure of the test IC. Also, since the block is used in a larger
schematic, the goal is to properly load the previous stage by
modelling the input impedance. The schematic of the complete
circuit is given in Fig. 4b, where the modelled circuit block is
labelled as BG. The remaining blocks, the bulk switch (BLKS)
and the output buffer (OB), are netlist blocks.
The architecture used for this test case is presented in Fig. 5.
In order to make the BG block model interchangeable within
the schematic, an approach similar to [36] is used to model
the input impedance. However, instead of modelling the value
of an impedance block as a function of the RF frequency and
amplitude, the modelled quantity is the input current waveform in the time-domain as a function of the input voltage.
Therefore, the input impedance is modelled using a voltage
controlled current source (VCCS). The output of the circuit is
modelled using a voltage controlled voltage source (VCVS). It
was shown experimentally that the presented architecture can
properly load the netlist block BLKS. Furthermore, another
black-box model that has the same architecture would be
properly loaded as well.
The operating conditions for the model are chosen to include
the frequency range between 30 MHz and 300 MHz, and the
forward power range from −20 dBm to 0 dBm. This design
space can be represented in a 2D plane.
The input impedance is modelled using an ESN with 20 neurons, while the output voltage is modelled using an ESN
with 35 neurons. The training dataset consists of a series of uniformly sampled time-domain waveforms with the
sampling time of 200 ps, containing approximately 2 ·
106 data points. The time required to train the ESN
is approximately 40 seconds using a machine with an
TM
R
Intel
Core i5-4460 CPU @ 3.2 GHz and 8 GB of RAM.
The achieved accuracy of the model is presented in
Fig. 6. Figs. 6a, 6b show the MSE of the input impedance
model, i.e. the modelled input current waveform IIN,BG and
the input voltage waveform VIN,BG in Fig. 4b. The MSE
for IIN,BG varies between 5.772·10−13 and 2.713·10−9 , and
between 1.658·10−7 and 1.348·10−3 for VIN,BG . It should be
noted that the input voltage VIN,BG is modelled implicitly,
through the interaction between the netlist of the BLKS block
and the VCCS of the BG block model.
Figs. 6c, 6d show the MSE of the modelled output voltage
of the BG block, and the relative error of the mean value
of the output voltage of the entire circuit. The mean value
of the bandgap reference output voltage is often used as a
failure criterion in the DPI test. The MSE for VOU T,BG varies
between 6.187·10−6 and 1.501·10−3 , and the relative error of
the mean output voltage varies between 0.009% and 4.644%.
In order to give proper interpretation of the MSE values, the
time-domain waveforms for the worst-case MSE points of the
modelled input impedance and the modelled output voltage
are shown in Fig. 7.
The simulation speed-up of the BG block model is presented
in Table I. The simulation times for the BG block without
the BLKS and OB blocks, Tmod,BG and Tmod,DU T for the
netlist and the model respectively, is evaluated by driving
the BG block using the input waveforms from file. All
simulations are run for 100 periods of the RF disturbance.
B. Discussion
The presented test case shows the achieved model accuracy
is comparable to the results reported in [36]. Compared to the
time required to build the model in [36], which is in the order
of several hours, the presented ESN model is trained extremely
fast.
The comparison between the simulation time for
the BG block netlist and the model (shown in columns Tref,BG
and Tmod,BG in Table I) shows an increase in the netlist
simulation time in the order of 5 times with increasing power
KVALIFIKACIJSKI DOKTORSKI ISPIT
MSE >10−9 ]
6
MSE
−3
>10 @
fR F 0
+] PRF>G%[email protected]
(a) The mean square error of the VCCS model: IIN,BG .
MSE
−3 >10 @
PRF>G%[email protected]
(b) The mean square error of the VCCS model: VIN,BG .
err >@
fR F 0
+] fR F 0
+] PRF>G%[email protected]
(c) The mean square error of the VCVS model: VOU T,BG .
fR F 0
+] PRF>G%[email protected]
(d) The error of the mean value of the output voltage: VBG .
Fig. 6: The MSE performance of the black-box model.
levels. This can be interpreted as the consequence of the
transistors-level models entering the nonlinear regime for
higher power levels, which increases the complexity of the
netlist. The ESN model, on the other hand, shows a minimal
increase in simulation time with increasing power levels.
This is an expected result because the model has the same
complexity regardless of the forward power level.
It should be noted that the model is built using the training
dataset obtained by uniformly sampling the frequency–power
design space. This simplistic approach is used to obtain an
initial estimation of the applicability of the ESN for building
circuit models for DPI simulations. As it can be seen in [36],
using the LOLA-Voronoi sampling algorithm [37] enables obtaining a training dataset that puts more focus on the nonlinear
regions of the design space, while avoiding undersampling the
linear regions.
V. F UTURE WORK
This section presents an overview of the modelling methods
that are related to the ESN, or a direct modification of the ESN.
The promises and trade-offs of these methods are discussed.
Their application for this research project is to be investigated
in future work.
A. Extreme learning machines
As it can be seen in the examples presented in Section III, the feedforward neural network is frequently used for
functional approximation. Having the universal approximation
property and a simple structure, circuit models based on the
feedforward neural network promise significant simulation
speed-up in cases where they are applicable.
The main drawback of the feedforward neural network are
the training algorithms that suffer from the problems described
in Section II-B. In [40] a training algorithm that promises to
solve this problem is proposed, called the extreme learning
machine (ELM). It is mathematically shown that single-layer
feedforward neural networks (SLFN) with random fixed input
weights and bias values retain the universal approximation
property of the SLFN with all trained weights, if the neuron
activation function is infinitely differentiable. Fixing the input
weights and bias values reduces the training algorithm to
a linear regression task on the output weights, making it
extremely efficient. This approach is similar to the algorithm
used in the ESN.
A comprehensive feasibility study of the ELM can be found
in [41], [42], and an overview of the ELM applications in
different problem domains is available in [43].
B. Support vector machines
The support vector regression machine (SVR) is a neural
network that uses kernel functions to transform the regression
problem to a higher dimensional space. The motivation is to
obtain a linear problem at the cost of increased dimensionality.
The main drawback of SVR is that the number of neurons
KVALIFIKACIJSKI DOKTORSKI ISPIT
IIN,BGP$
02'(/
'(6,5(' (5525
0+]
G%P
error—$
7
tQV
TABLE I: The simulation times required for simulating 100 periods. Columns Tref,BG and Tref,DU T are the
simulation times obtained using the transistor-level models of
the BG block and of the entire DUT, respectively. Columns
Tmod,BG and Tmod,DU T are the simulation times obtained
using the black-box model of the BG block.
fRF
[MHz]
PRF
[dBm]
Tref,BG
[s]
Tmod,BG
[s]
Tref,DU T
[s]
Tmod,DU T
[s]
30
-20
-10
0
5.9
20.4
47.8
4.6
4.9
6.5
13.3
23.0
30.6
14.2
18.9
26.8
100
-20
-10
0
7.9
17.3
40.3
4.0
5.2
7.1
16.6
26.0
34.0
15.8
20.8
29.5
300
-20
-10
0
6.6
10.7
18.9
6.7
4.5
8.8
14.2
21.7
30.1
14.3
18.1
28.1
(a) Input current. fRF = 300 MHz, PRF = 0 dBm.
0+]
G%P
VOUT,BG9
02'(/
'(6,5(' (5525
errorP9
tQV
(b) Output voltage. fRF = 30 MHz, PRF = 0 dBm.
Fig. 7: The time-domain responses of the input current and
output voltage signals of the modelled IC block taken at their
worst-case MSE points, respectively. The instantaneous error
is shown.
(support vectors) depends on the size of the training dataset
due to the architecture of the SVR. In the case of the echo state
network the number of neurons is not inherently dependent
on the size of the training dataset. In [44] the SVR approach
is modified to produce recurrent models in the time-domain
with sparse solutions. The proposed approach is evaluated on
standard benchmark tests for neural networks, presented in
Section III.
In future work the applicability of the SVR to the modelling that are not successfully handled by the ESN will be
investigated, e.g. the startup sequence of a bandgap reference
circuit.
C. Time-warping invariant neural networks
It was shown during this research project that it is extremely
difficult to model signals that have fast transients followed by
long periods of “flat” signal values, e.g. the startup sequence
of a bandgap reference. The discrete-time implementation of
any ANN implicitly assumes the timestep between every two
samples is equal. The above mentioned signals have to be
sampled with very short timesteps due to the fast transients,
at the price of having extremely large datasets due to the long
flat signals.
Using non-uniform sampling rates is referred as time-warping. Examples of implementing neural networks that are
able to model such datasets can be found in the literature. E.g. [45] introduces the time-warping invariant neural
network (TWINN). This concept is based on replacing the
implicit information about the timestep by the norm of the
change in the signal between two consecutive non-uniform
timesteps.
In [46] the time-warping invariant echo state network (TWIESN) is introduced. Following the principle shown
in [45], the ESN is modified for pattern recognition of highly
time-warped patterns. So far the TWI concept has not been
reported for modelling non-uniformly sampled time-series,
which will be investigated.
D. X-parameters
Artificial neural networks are used as time-domain
black-box models. In the frequency domain, the most commonly used black-box models are the S-parameters. As an
already mature modelling method, many books cover the
theory behind S-parameters, e.g. [47]. The most appealing
property of S-parameters is that they can be readily obtained
by measurements using the vector network analyser (VNA),
and subsequently used as a model in circuit simulators, e.g.
Keysight Advanced Design System (ADS).
The main drawback of S-parameters is that they can be
used to describe only linear devices. Many methods have been
proposed to extend the S-parameters to nonlinear devices,
e.g. “hot” S-parameters [48], however without much success.
In [48], the X-parameters are introduced as a true extension
of the S-parameters for nonlinear devices. The X-parameters
represent a mathematical framework for including the influence of each harmonic component on every other harmonic
component, a phenomenon that occurs in nonlinear systems
that is not included in S-parameters.
Being a relatively new concept, and being proprietary to
Keysight, the X-parameters are not yet accepted as an industry
standard. Furthermore, the application of X-parameters in [48]
KVALIFIKACIJSKI DOKTORSKI ISPIT
is mainly for power amplifier design. However, the mathematical background behind X-parameters shows promise
to be used as the theoretical basis for modelling the input
impedance of circuits for EMC simulations in the frequency
domain, analogously to the time-domain approach presented
in Section IV.
8
reference block for computer simulations of the direct power
injection EMC test, are encouraging. Finally, the guidelines
for the future work are given.
ACKNOWLEDGMENT
This research is funded by ams AG.
E. Echo state network modifications
In [49] the decoupled echo state network (DESN) is introduced to tackle the stochastic nature of generating the ESN.
The main idea is to use multiple ESN reservoirs in parallel
with a lateral inhibition mechanism that reduces the correlation
between the different reservoirs in order to enable the generation of more rich dynamics. The reservoir sizes used are in the
order of 100, which is plausible for use in circuit simulators,
and the experimental results show very good performance.
In [22] an augmented echo state network for use in adaptive
filtering is introduced, that can process complex-valued signals. The augmented ESN shows better overall performance
than the standard ESN also for the real-valued signals in the
one-step-ahead prediction tests.
F. Concluding remarks
The next steps of this research project should include
investigating two paths. The first one is to explore how to
build the EMC models directly from DPI measurements. This
requirement follows from the fact that the circuit simulations
for high forward power levels of the RF disturbance invariably
break, or produce non-verified results. The challenge in this
approach is to incorporate the DPI measurements, which
are the measurements of power levels and do not include
the phase information, in the ESN model building method.
The second path is to explore the applicability of alternative
neural network architectures overviewed in this paper for
the EMC simulations. In particular, the circuit models should
not only model the RF disturbance well, but also the normal
functionality of the circuit as well. The example of modelling
the startup sequence of a bandgap reference circuit, mentioned
in Section V-C, showed to be a very difficult modelling task
during this research project.
VI. C ONCLUSION
In this paper an overview of the state-of-the-art in black-box
modelling is given. The theory of using artificial neural
networks for building black-box models is presented, including
generic model building methods and an overview of the
training algorithms. Implementing the obtained models in
commercially available circuit simulators is discussed. A more
detailed overview of the echo state network is presented, due
to its favourable properties that include an extremely fast and
robust training method, excellent generalisation properties, and
conceptual simplicity. Examples from the literature of using
echo state networks and artificial neural networks in general
and in engineering applications are presented. Initial results of
this research project, specifically using the echo state network
to produce an interchangeable circuit model of a bandgap
R EFERENCES
[1] J. Wood and D. E. Root, Fundamentals of Nonlinear Behavioral
Modeling for RF and Microwave Design, ser. Artech House microwave
library. Artech House, 2005.
[2] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks, vol. 4, no. 2, pp. 251 – 257, 1991.
[3] P. Baldi and K. Hornik, “Universal Approximation and Learning of
Trajectories Using Oscillators,” in 1995 Advances in Neural Information
Processing Systems (NIPS), Nov 1995, pp. 451–457.
[4] M. Makula, M. Cernansky, and L. Benuskova, “Approaches Based
on Markovian Architectural Bias in Recurrent Neural Networks,” in
SOFSEM 2004: Theory and Practice of Computer Science, ser. Lecture
Notes in Computer Science. Springer Berlin Heidelberg, 2004, vol.
2932, pp. 257–264.
[5] M. Cernansky and M. Makula, “Feed-forward Echo State Networks,” in
2005 IEEE Int. Joint Conf. on Neural Networks (IJCNN ), vol. 3, July
2005, pp. 1479–1482.
[6] F. Dan Foresee and M. T. Hagan, “Gauss-Newton approximation to
Bayesian learning,” in 1997 Int. Conf. on Neural Networks, vol. 3, Jun
1997, pp. 1930–1935.
[7] M. T. Hagan and M. B. Menhaj, “Training feedforward networks with
the Marquardt algorithm,” IEEE Trans. Neural Netw., vol. 5, no. 6, pp.
989–993, Nov 1994.
[8] A. Atiya and A. Parlos, “New Results on Recurrent Network Training:
Unifying the Algorithms and Accelerating Convergence,” IEEE Trans.
Neural Netw., vol. 11, no. 3, pp. 697–709, May 2000.
[9] H. Khalil, Nonlinear Systems, ser. Pearson Education. Prentice Hall,
2002.
[10] H. Jaeger, “The “echo state” approach to analysing and training recurrent
neural networks - with an Erratum note,” Fraunhofer Institute for
Autonomous Intelligent Systems, Tech. Rep., Jan 2001. (rev. 2010.).
[11] S. Boyd and L. Vandenberghe, Convex Optimization.
Cambridge
University Press, 2004.
[12] Q. Song, X. Zhao, Z. Feng, and B. Song, “Recursive Least Squares
Algorithm with Adaptive Forgetting Factor Based on Echo State Network,” in 2011 9th World Congr. on Intelligent Control and Automation
(WCICA), June 2011, pp. 295–298.
[13] A. A. Ferreira, T. B. Ludermir, and R. R. de Aquino, “An approach to
reservoir computing design and training,” Expert Systems with Applications, vol. 40, no. 10, pp. 4172–4182, 2013.
[14] J. Fan and M. Han, “Online designed of Echo State Network based
on Particle Swarm Optimization for system identification,” in 2011 2nd
Int. Conf. on Intelligent Control and Information Processing (ICICIP),
vol. 1, July 2011, pp. 559–563.
[15] P. A. Duran, A Practical Guide to Analog Behavioral Modeling for IC
System Design. Norwell, MA, USA: Kluwer Academic Publishers,
1998.
[16] K. Suzuki, A. Nishio, A. Kamo, T. Watanabe, and H. Asai, “An
Application of Verilog-A to Modeling of Back Propagation Algorithm
in Neural Networks,” in 2000 43rd IEEE Midwest Symp. on Circuits
and Systems, vol. 3, 2000, pp. 1336–1339 vol.3.
[17] I. B. Yildiz, H. Jaeger, and S. J. Kiebel, “Re-Visiting the Echo State
Property,” Neural Networks, vol. 35, pp. 1–9, 2012.
[18] A. Rodan and P. Tino, “Minimum Complexity Echo State Network,”
IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 131–144, Jan 2011.
[19] Y. Pan and J. Wang, “Model Predictive Control of Unknown Nonlinear
Dynamical Systems Based on Recurrent Neural Networks,” IEEE Trans.
Ind. Electron., vol. 59, no. 8, pp. 3089–3101, Aug 2012.
[20] V. Ceperic and A. Baric, “Reducing Complexity of Echo State Networks
with Sparse Linear Regression Algorithms,” in 2014 16th Int. Conf. on
Computer Modelling and Simulation (UKSim), Mar 2014, pp. 26–31.
[21] J. Xu, J. Yang, F. Liu, Z. Zhang, and A. Shen, “Echo state networks
based method for harmonic extraction in shunt active power filters,” in
2011 6th Int. Conf. on Bio-Inspired Computing: Theories and Applications (BIC-TA), Sept 2011, pp. 135–139.
KVALIFIKACIJSKI DOKTORSKI ISPIT
[22] Y. Xia, B. Jelfs, M. Van Hulle, J. Principe, and D. Mandic, “An
Augmented Echo State Network for Nonlinear Adaptive Filtering of
Complex Noncircular Signals,” IEEE Trans. Neural Netw., vol. 22, no. 1,
pp. 74–83, Jan 2011.
[23] T. Strauss, W. Wustlich, and R. Labahn, “Design Strategies for Weight
Matrices of Echo State Networks,” Neural Comput., vol. 24, no. 12, pp.
3246–3276, Dec. 2012.
[24] M. C. Ozturk, D. Xu, and J. C. Principe, “Analysis and Design of Echo
State Networks,” Neural Comput., vol. 19, no. 1, pp. 111–138, Jan. 2007.
[25] B. Zhang, D. Miller, and Y. Wang, “Nonlinear System Modeling With
Random Matrices: Echo State Networks Revisited,” IEEE Trans. Neural
Netw. Learn. Syst., vol. 23, no. 1, pp. 175–182, Jan 2012.
[26] R. F. Reinhart and J. J. Steil, “Regularization and stability in reservoir
networks with output feedback,” Neurocomputing, vol. 90, pp. 96–105,
2012.
[27] D. Sussillo and L. Abbott, “Transferring Learning from External to
Internal Weights in Echo-State Networks with Sparse Connectivity,”
PLoS ONE, vol. 7, no. 5, p. e37372, 05 2012.
[28] D. Prokhorov, “Echo state networks: appeal and challenges,” in
2005 IEEE Int. Joint Conf. on Neural Networks (IJCNN), vol. 3, July
2005, pp. 1463–1466.
[29] G. K. Venayagamoorthy and B. Shishir, “Effects of spectral radius
and settling time in the performance of echo state networks,” Neural
Networks, vol. 22, no. 7, pp. 861–863, 2009.
[30] G. Fette and J. Eggert, “Short Term Memory and Pattern Matching with
Simple Echo State Networks,” in Artificial Neural Networks: Biological
Inspirations ICANN 2005, ser. Lecture Notes in Computer Science.
Springer Berlin Heidelberg, 2005, vol. 3696, pp. 13–18.
[31] J. Dai, G. Venayagamoorthy, R. Harley, and K. Corzine, “Indirect
adaptive control of an active filter using Echo State Networks,” in
2010 IEEE Energy Conversion Congr. and Expo. (ECCE), Sept 2010,
pp. 4068–4074.
[32] Y. Cao, I. Erdin, and Q.-J. Zhang, “Transient Behavioral Modeling
of Nonlinear I/O Drivers Combining Neural Networks and Equivalent
Circuits,” IEEE Microw. Compon. Lett., vol. 20, no. 12, pp. 645–647,
Dec 2010.
[33] I/O Buffer Information Specification (IBIS), Electronic Industries Alliance Std. IBIS Version 6.0, Sept 2013.
[34] I. Stievano, C. Siviero, I. Maio, and F. Canavero, “Guaranteed Locally-Stable Macromodels of Digital Devices via Echo State Networks,”
in 2006 IEEE Electrical Performance of Electronic Packaging, Oct 2006,
p. 65-68.
[35] I. Chahine, M. Kadi, E. Gaboriaud, A. Louis, and B. Mazari, “Characterization and Modeling of the Susceptibility of Integrated Circuits to
Conducted Electromagnetic Disturbances Up to 1 GHz,” IEEE Trans.
Electromagn. Compat., vol. 50, no. 2, pp. 285–293, May 2008.
[36] C. Gazda, D. Ginste, H. Rogier, I. Couckuyt, T. Dhaene, K. Stijnen, and
H. Pues, “Harmonic balance surrogate-based immunity modeling of a
nonlinear analog circuit,” IEEE Trans. Electromagn. Compat., vol. 55,
no. 6, pp. 1115–1124, Dec 2013.
[37] K. Crombecq, L. De Tommasi, D. Gorissen, and T. Dhaene, “A
Novel Sequential Design Strategy for Global Surrogate Modeling,” in
2009 Winter Simulation Conf. (WSC), Dec 2009, pp. 731–742.
[38] Integrated circuits - Measurement of electromagnetic immunity 150 kHz
to 1 GHz - Part 4: Direct RF power injection method, International
Electrotechnical Commission Std. IEC 62 132-4, 2006.
[39] A. Alaeldine, R. Perdriau, M. Ramdani, J. Levant, and M. Drissi, “A
Direct Power Injection Model for Immunity Prediction in Integrated
Circuits,” IEEE Trans. Electromagn. Compat., vol. 50, no. 1, pp. 52–62,
Feb 2008.
[40] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine:
Theory and applications,” Neurocomputing, vol. 70, no. 13, pp. 489–501,
2006.
[41] X. Liu, S. Lin, J. Fang, and Z. Xu, “Is Extreme Learning Machine
Feasible? A Theoretical Assessment (Part I),” IEEE Trans. Neural Netw.
Learn. Syst., vol. 26, no. 1, pp. 7–20, Jan 2015.
[42] S. Lin, X. Liu, J. Fang, and Z. Xu, “Is Extreme Learning Machine
Feasible? A Theoretical Assessment (Part II),” IEEE Trans. Neural Netw.
Learn. Syst., vol. 26, no. 1, pp. 21–34, Jan 2015.
[43] E. Cambria, G.-B. Huang, L. L. C. Kasun, H. Zhou, C. M. Vong, J. Lin,
J. Yin, Z. Cai, Q. Liu, K. Li, V. C. Leung, L. Feng, Y.-S. Ong, M.-H.
Lim, A. Akusok, A. Lendasse, F. Corona, R. Nian, Y. Miche, P. Gastaldo,
R. Zunino, S. Decherchi, X. Yang, K. Mao, B.-S. Oh, J. Jeon, K.-A. Toh,
A. B. J. Teoh, J. Kim, H. Yu, Y. Chen, and J. Liu, “Extreme Learning
Machines [Trends & Controversies],” IEEE Intell. Syst., vol. 28, no. 6,
pp. 30–59, Nov 2013.
9
[44] V. Ceperic, G. Gielen, and A. Baric, “Recurrent sparse support vector
regression machines trained by active learning in the time-domain,”
Expert Systems with Applications, vol. 39, no. 12, pp. 10 933–10 942,
2012.
[45] G.-Z. Sun, H.-H. Chen, and Y.-C. Lee, “Time Warping Invariant Neural
Networks,” in Advances in Neural Information Processing Systems 5.
Morgan-Kaufmann, 1993, pp. 180–187.
[46] M. Lukosevicius, D. Popovici, H. Jaeger, and U. Siewert, “Time Warping
Invariant Echo State Networks,” Jacobs University Bremen, Technical
Report 2, May 2006.
[47] D. Pozar, Microwave Engineering. Wiley, 2004.
[48] D. E. Root, J. Verspecht, J. Horn, and M. Marcu, X-Parameters: Characterization, Modeling, and Design of Nonlinear RF and Microwave
Components. Cambridge University Press, 2013.
[49] Y. Xue, L. Yang, and S. Haykin, “Decoupled Echo State Networks with
Lateral Inhibition,” Neural Networks, vol. 20, no. 3, pp. 365–376, 2007.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement