Unitary Hessen b erg

Unitary Hessen b erg
Unitary Hessenberg matrices and the generalized
Parker-Forney-Traub algorithm for inversion of
Szeg
o-Vandermonde matrices
V.Olshevsky
y
Abstract
It is well-known that the Horner polynomials (sometimes called the associated
polynomials)
describe the structure of the inverses of Vandermonde matrices V (x) =
h
i
j
1
xi . This description led to the fast O(n2) inversion algorithm discovered independently by Parker, Forney, Traub and many others. In this paper we show how
the generalized Horner polynomials
h dene the
i structure of the inverses of polynomialVandermonde matrices VP (x) =
Pj 1 (xi) , and use this description to generalize to
VP (x) the Parker-Forney-Traub inversion algorithm. We show that in the case when
the polynomials fPk (x)g involved in VP (x) are the Szego polynomials, the properties of
the corresponding unitary Hessenberg matrix allow us a dramatic simplication, leading to fast O(n2 ) computational procedures for inversion of what we suggest to call
Szego-Vandermonde matrices VP (x).
1
Introduction
Vandermonde matrices of the form
2
3
1
x1 x21 xn1 1
6
... 77
V (x) = 64 ... ... ...
(1.1)
5
2
n
1
1 xn xn xn
are classical, and explicit expressions for their determinants and inverses are well-known.
The structure in (1.1) can be exploited to design fast algorithms. The algebraic complexity
of such fast algorithms is typically by an order-of-magnitude less than the one of the standard (structure-ignoring) methods. For example, standard matrix inversion methods require
O(n3) operations. However, the structure of V (x) 1 described by Kowalewski in [K32] can
This work was supported in part by NSF contract CCR 9732355.
The views and conclusions contained
in these documents are those of the author(s) and should not be interpreted as necessarily representing the
oÆcial policies or endorsements, either expressed or implied, of the National Science Foundation, or the U.S.
Government.
y Department of Mathematics and Computer Science, Georgia State University, Atlanta, GA 30303,
E-mail: volshevsky@cs.gsu.edu Web: http://www.cs.gsu.edu/~matvro
1
be used to design a fast O(n2) algorithm. This fast inversion algorithm has been re-derived
in the mathematical and engineering literature several times, and now it is usually associated
with the names of Parker [P64], Forney [F66] and Traub [T66] (see also [Wertz65], [K69]).
Classical Vandermonde matrices (1.1) appear in polynomial computations exploiting the
monomial basis f1; x; x2 ; : : : ; xn 1 g. An alternative use of orthogonal on a real interval
polynomials gives rise to the more general three-term Vandermonde matrices,
2
3
r0 (x1 ) r1 (x1 ) rn 1 (x1 )
6 r (x ) r (x ) r (x ) 7
6 0 2
1 2
n 1 2 7
VR (x) = 66 ..
(1.2)
.
... 775 ;
.
4
.
.
r0 (xn ) r1 (xn ) rn 1 (xn )
where the polynomials fr0(x); r1 (x); : : : ; rn 1(x)g satisfy three-term recurrence relations. A
good performance record of fast algorithms for Vandermonde matrices attracted much attention in the numerical linear algebra literature1, and the Parker-Forney-Traub algorithm
was generalized to invert Chebyshev-Vandermonde matrices [GO94], and three-term Vandermonde matrices [CR93].
Along with the above real-line settings (giving rise to three-term Vandermonde matrices),
it is important to consider the case when the inner product is dened on the unit circle, i.e.,
Z
1
i
i 2
(1.3)
< p(x); q (x) >=
2 p(e ) [q(e )] w ()d:
Polynomials = fk (x)g orthogonal with respect to (1.3) appear in various signal processing
applications, and they are called the Szego polynomials. It is well known that the Szego
polynomials are completely described by the two-term recurrence relations [GS58], [G48],
" #
" #
#
"
#"
#"
#
" #
#
1
k+1
1 0 #k (x) ;
0 (x) = 1 1 ;
k+1 (x) = 1
k+1
1
0 x k (x)
k+1 (x)
0 (x)
0 1
k
(1.4)
where the numbers f0 ; 1; : : : ; ng, are called reection coeÆcients (the
names
parcor
coq
eÆcients and Schur parameters are also in use). The numbers k = 1 jk j2 are called
the complementary parameters (k := 1 if k = 1), and #k (x) = xk [( x1 )]. The reection
coeÆcients dene the Hessenberg matrix
2
6
6
6
6
C = 66
6
6
4
10
1
2 1 0
2 1
2
32 1 0
3 2 1
3 1
n 1 n 2 :::1 0
n 1 n 2 :::2 1
n 1 n 2 :::3 2
3
n n 1 :::1 0
n n 1 :::2 1 7
7
n n 1 :::3 2 7
7
7
...
7
7
7
nn 1 n 2 5
nn 1
0
...
...
...
(1.5)
3
...
...
...
n 1 n 2
0
0
n 1
1 Interestingly, most of excellent numerical results were reported for the Bj
orck-Pereyra algorithm for
solving Vandermonde linear equations (see, e.g., [BP70], [GL89], [BKO00])). In contrast, the fast inversion
algorithm for Vandermonde matrices has been incorrectly regarded as inaccurate, mostly because of the poor
numerical performance of its version of [T66]. It was numerically demonstrated in [GO96] that the Parker
version [P64] typically produces an excellent accuracy.
2
which has many nice properties, for example, it is well-known that C diers from unitary
only in the last column. Such almost-unitary Hessenberg matrices have been studied in signal
processing literature, because they describe the state-space structure for lattice digital lters,
see, e.g., [ML80], [KP83], [TKH83], [K85]. The nice structure of C has been independently
studied in numerlical linear algebra literature, and exploiting it allowed to develop eÆcient
algorithms for several problems involving the Szego polynomials, see [G82], [G86], [GR90],
[AGR93], [ACR96] among others. We would like to also mention that in the operator theory
literature, see, e.g., [C84], [FF89], the matrices of the form (1.5) are associated with the
Naimark dilation.
In a parallel paper [O98] we introduce the Horner-Szego polynomials using the language
of discrete transmission lines, and then use them for eigenvalue computations. In this paper we use the structure of almost-unitary Hessenberg matrices to study the Horner-Szego
polynomials. We then use them to generalize the Parker-Forney-Traub algorithm to what
we call Szego-Vandermonde matrices, i.e., the matrices of the form (1.2) involving the Szego
polynomials.
The paper is structured as follows. In the next section we recall the conventional ParkerForney-Traub algorithm, providing its new interpretation in terms of the corresponding
companion matrices. This interpretation serves as a starting point to formulate in Sec. 3 the
generalized Parker-Forney-Traub algorithm for inversion of general polynomial Vandermonde
matrices VR (x), in which polynomials only satisfy deg rk (x) = k. This algorithm is designed
via a simple operation on the corresponding confederate matrix (i.e., the Hessenberg matrix
generalizing the companion matrix). In section 4 we specify the generalized Parker-ForneyTraub algorithm to Szego-Vandermonde matrices, for which the corresponding confederate
matrix has the form shown in (1.5). This allows to achieve a favorable cost O(n2). Some
conclusions are oered in the nal section.
2
Associated (Horner) polynomials and the ParkerForney-Traub algorithm
Let P be the system of n + 1 polynomials P =
f1
( )g, where ( ) = 0 + b1 x + ::: + bn 1 xn 1 + bn xn. Then the rst
divided dierence of b(x) has the Hankel form
n
b(x) b(y ) X
= b (xi 1 + xi 2 y + ::: + x yi 2 + xi 1 ) =
(2.1)
B (x; y ) =
2.1. Classical Horner polynomials.
; x; x2 ; :::; xn 1 ; b x
bx b
x y
i=1
i
2
h
= 1 x x2 6
6
i 6
6
n
1
x
6
6
6
4
b1 b2 b3 bn
b2 b3 bn 0
.
b3 bn . . . ..
... . . . . . .
bn 0 3
...
0
3 2
7
7
7
7
7
7
7
5
6
6
6
6
6
6
6
4
1
y
y2
...
yn 1
3
7
7
7
7
7=
7
7
5
2
h
6
6
i 6
xn 1 666
6
4
p^n 1 (y )
p^n 2 (y )
3
7
7 n 1
7 X
7
7=
xi
7
7 i=0
5
...
p^n 1 i(y);
(2.2)
p^1 (y )
p^0 (y )
where the polynomials P^ = fp^0 (x); p^1(x); :::; p^n 1(x); p^n(x)g, with p^n(x) = b(x) satisfy the
recurrence relations
p^0 (x) = bn ;
p^i (x) = x p^i 1 (x) + bn i ;
(2.3)
and they are usually coined with the name of Horner, who used them for solving a single nonlinear equation. However, as was noted in [C11], the Horner's method was anticipated by RuÆni, and moreover Horner himself pointed out in [H1819], that these polynomials were known to Lagrange [L1775]. However, all of the above men had been anticipated by Chinese mathematicians, see e.g. [C11]. For these historical reasons in [T66]
polynomials P^ = fp^0(x); p^1 (x); :::; p^n 1(x); p^n(x)g were called in [T66] the associated with
P = f1; x; x2 ; :::; xn 1 ; b(x)g polynomials.
2.2. The Parker-Forney-Traub inversion algorithm. Traub used the simplicity of
the recursion (2.3) to derive in [T66] a fast O(n2) algorithm for inversion of Vandermonde
matrices. Before describing his result, let us introduce the necessary notations. Denote by
2
3
r0 (x1 ) r1 (x1 ) rn 1 (x1 )
6 r (x ) r (x ) r (x ) 7
6 0 2
1 2
n 1 2 7
VR (x) = 66 ..
(2.4)
.
... 775
.
4
.
.
r0 (xn ) r1 (xn ) rn 1 (xn )
a polynomial Vandermonde matrix corresponding to the set of nodes x = (x1 ; x2; :::; xn)
and to the system of polynomials R = fr0 (x); r1(x); :::; rn(x)g. Let I~ be the antidiagonal
identity matrix, and superscript T means transpose. With these notations the equality (2.2)
immediately implies
i
h
VP (x) I~ VP^ (x)T = b(xx) xb(x ) 1i;j n ;
(2.5)
where the diagonal entries are understood as b0(xi ) ( i = 1; 2; :::; n ). Observe that (2.5) is
valid for arbitrary polynomial b(x), and if we additionally set
= 1 x x2 i
j
i
then
where
j
b(x) = ni=1 (x xi );
(2.6)
VP (x) 1 = I~ VP^T diag(c1 ; c2 ; :::; cn);
(2.7)
ci = b0 (xi ) =
1
(xk xi) :
n
k=1
k6=i
Traub used (2.7) to formulate the following eÆcient inversion algorithm.
4
(2.8)
The Parker-Forney-Traub algorithm.
1 ). Compute the numbers ci by (2.8).
2 ). Compute the coeÆcients bi ( i = 0; 1; :::; n ) of the polynomial b(x) in (2.6).
3 ). Compute the entries of VP^ (x) using recursion (2.3).
4 ). Multiply the matrices on the right hand side of (2.7).
These computations require performing only 6n2 arithmetic operations, achieving a favorable eÆciency as compared to the expensive complexity O(n3) of the Gaussian elimination.
A key point that makes this speed-up possible is that for the monomial system P , the recurrence relations (2.3) for the Horner polynomials P^ remain sparse. Our next goal is to
carry over the Parker-Forney-Traub algorithm to the more general polynomial-Vandermonde
matrices VR (x); to this end we need to dene generalized Horner polynomials for an arbitrary
polynomial system R, and to obtain for them an analogue of (2.3). This will be achieved by
using a useful matrix interpretation of (2.3), given next.
2.3. A confederate matrix interpretation. Let polynomials R = fr0 (x); r1 (x) ; :::;
rn(x)g be specied by the recurrence relations
rk (x) = k x rk 1 (x) ak 1;k rk 1 (x) ak 2;k rk 2 (x) ::: a0;k r0 (x):
(2.9)
Following [MB79], dene for the polynomial
b(x) = b0 r0 (x) + b1 r1 (x) + ::: + bn 1 rn 1 (x) + bn rn (x)
(2.10)
its confederate matrix
2 a01
6 11
6 6 1
6 0
6
CR (b) = 666 0
6
6 ..
6 .
4
a02
2
a12
2
1
2
a03
3
a13
3
a23
3
a0;n
a1n;n
a2n;n
n
1
1
1
n
n
n
3
bb
bb
bb
0
7
7
7
7
7
7
7
7
7
7
7
5
n
1
n
2
...
(2.11)
0 ... ...
... . . . . . . . . .
...
0 0 0 1 1 a 1 1 b b 1
with respect to the polynomial system R. We refer to [MB79] for many useful properties of
the confederate matrix and only recall here that det(xI CR (b)) = b(x)=(0 1 ::: n bn),
and that similarly, the characteristic polynomial of the k k leading submatrix of CR (b) is
equal to rk (x)=0 1 ::: k .
In the simplest case P = f1; x; x2; :::; xn 1 ; b(x)g, the confederate matrix CP (b) reduces
to the well known companion matrix
2
0 0 0 b0 3
6 1 0 0
b1 77
6
6
. ... 777
CP (b) = 666 0 1 . . . ..
(2.12)
7
.
6 ..
7
.
. . 0 .. 5
4 .
0 0 1 bn 1
n
n
5
;n
n
n
n
n
n
of the monic polynomial b(x) = b0 + b1 x + ::: + bn 1 xn 1 + xn. Now observe that the
recurrence relations (2.3) for the Horner polynomials P^ = fp^0(x); :::; p^n(x)g give the following
nice structure for the corresponding confederate matrix :
2
6
6
6
CP^ (^pn ) = 666
6
4
bn 1
1
0
...
0
bn 2
0
1
b1 b0 3
0 0 777
. . . ...
... 0
0 1
0
...
0
7
7:
7
7
5
(2.13)
Equalities (2.12) and (2.13) imply the following statement.
The confederate matrix of the monomials P = f1; x; x2 ; :::; xn 1 ; b(x)g
and that of the Horner polynomials P^ = fp^0 (x); :::; p^n (x)g are related by
Proposition 2.1
CP^ (^pn ) = I~ CP (b)T I~
(2.14)
where I~ is the antidiagonal identity matrix.
In the next section we shall use (2.14) to extend the concept of the Horner polynomials to
arbitrary polynomial system R.
2.4. Generalized Horner polynomials and displacement structure. In [KO94]
we used (for arbitrary given polynomials R = fr0(x); ::; rn(x)g) the generalized Horner polynomials R^ = fr^0(x); :::; r^n(x)g that were specied by their confederate matrix
CR^ (^rn) = I~ CR (rn )T I~;
(with r^n(x) = rn(x)):
(2.15)
Such polynomials R^ have been found to be useful to describe the structure of the inverses
of the more general polynomial Vandermonde-like matrices S that are dened as having low
displacement rank
rank(diag(x1 ; :::; xn) S S CR (rn))
The motivation for the above denition can be inferred from the easily veried fact that the
displacement rank of VR(x) is one :
2
3
rn (x1 ) h
i
6
7
diag(x1 ; :::; xn) VR(x) VR (x) CR (rn) = 64 ... 75 0 0 1 :
(2.16)
rn (xn )
It can be seen that the displacement rank of a matrix is essentially inherited under inversion :
rank(diag(x1 ; :::; xn) S S CR(rn)) = rank(diag(x1; :::; xn) (S T I~) (S T I~) CR^ (rn)):
n
Briey, the passage from S to (S T I~) corresponds to the passage from polynomials R to
the generalized Horner polynomials R^. This fact allowed us to to obtain in [KO94] explicit
inversion formulas for polynomial Vandermonde-like matrices in terms of generalized Horner
polynomials.
6
3
The generalized Parker-Forney-Traub algorithm
In this section we show how the generalized Horner polynomials R^ dene the structure of
VR (x) 1 . To this end we an auxiliary result, given next.
3.1. Change of basis. Along with a polynomial system Q = fq0 (x); q1 (x):::; qn (x)g
(satisfying deg qk (x) = k) consider another system R = fr0(x); r1 (x); :::rn(x)g with rn(x) =
qn (x) and
rk (x) = s0;k q0 (x) + s1;k q1 (x) + ::: + sk;k qk (x);
sk;k 6= 0:
(3.1)
Clearly,
h
i
h
i
q0 (x) q1 (x) q2 (x) qn 1 (x) SQR = r0 (x) r1 (x) r2 (x) rn 1 (x) ; (3.2)
and
VQ(x) SQR = VR (x);
(3.3)
where SQR is an upper triangular matrix whose columns are formed from the coeÆcients of
the polynomials in (3.1). According to [MB79] we have
CR (rn) = SQR1 CQ(qn ) SQR :
(3.4)
From here and (2.15) it follows immediately that
CR^ (^rn ) = (I~SQRT I~) 1 CQ^ (^qn ) (I~SQRT I~):
(3.5)
^ R; R^ can be depicted
Briey, the similarity matrices for the four polynomial systems P; P;
as follows ;
! R
#
~I S I~
P^
! R^
P
#
SP R
T
PR
(3.6)
Therefore (3.3) and(3.6) allow us to deduce from (2.7) the
more general formula for the inverse of an arbitrary polynomial-Vandermonde matrix :
VR (x) 1 = I~ VR^T diag(c1 ; :::; cn);
(3.7)
where ck are given by (2.8). This formula allows us to extend the Parker-Forney-Traub
algorithm to invert an arbitrary polynomial-Vandermonde matrix VR (x).
3.3. Inversion algorithm. To invert VR (x) by (3.7) all we need is
Q
(a) to compute, for R = fr0 (x); :::; rn 1 (x); b(x)g with b(x) = nk=1 (x xk ) the matrix CR (b).
To this end observe that the matrix CR(b) does not depend upon the last polynomial in
R, and in particular CR (b) = CR (b), where R = fr0 (x); :::; rn 1 (x); x rn 1 (x)g. Thus
to obtain CR (b) we need to nd the coeÆcients of b(x) decomposed in the basis R ,
which can be done recursively by starting with rn(0) (x) = 1 and by updating rn(k+1)(x) =
(x xk+1) rn(k)(x),
(b) to compute, using (2.15) the entries of VR^ (x).
3.2. Inversion formula.
7
A convenient procedures for doing these two are provided in the next two simple lemmas.
P
Let R = fr0 (x); :::; rn (x)g be givenPby (2.9), and f (x) = ki=1 ai ri (x),
+1 b r (x) can be computed by
1. Then the coeÆcients of x f (x) = ki=1
i i
Lemma 3.1
where k < n
2
b0
6 .
6 ..
6
6
6 bk+1
6
6 0
6
6 .
6 ..
4
3
2
7
7
7 "
7
7
7=
7
7
7
7
5
CR (rn )
0 0
1
n
0
0
0
Proof. It can be easily checked that
h
x r0 (x) r1 (x)
rn(x)
i
h
r0 (x) r1 (x)
6
6
#6
6
6
6
6
6
6
6
4
a0
.. 7
. 7
7
7
7
7
0 777
.. 7
. 5
ak
(3.8)
0
i
rn(x) h
3
"
CR (rn )
0 0
i
1
n
0
0
#
= 0 0 x rn(x) :
Multiplying the latter equation by the column of the coeÆcients we obtain (3.8).
Let R = fr0 (x); :::; rn (x)g be given by (2.9), Then (2.15) translates into the
following recursion for the generalized Horer polynomials
Lemma 3.2
R^ 0 (x) = ^0 ;
R^k (x) = ^k x R^ k 1 (x) a^0;k R^ 0 (x) a^1;k R^1 (x) ::: a^k 1;k R^k 1 (x);
where
and
^k = n k ;
a^k;j =
n j
a
n k n
j;n k
(3.9)
(k = 0; 1; :::; n);
(k = 0; 1; :::n 1; j = 1; 2; :::; n):
(3.10)
These two auxiliary statements allows us to write down the following algorithm.
The generalized Parker-Forney-Traub algorithm.
INPUT :
n numbers x = (x1 ; :::; xn ) and n polynomials fr0 (x); r1 (x); :::; rn 1 (x)g, specied
by the coeÆcients of the recurrence relations (2.9).
OUTPUT : The entries of VP (x) 1 .
STEPS :
1 ). Compute the numbers ci by (2.8).
2 ). Compute the coeÆcients in b(x) = Qnk=1(x xk ) = n x pn 1(x) an
::: a0;n p0 (x) by
8
1;n pn 1 (x)
h
(2 ).1 )) Set a(0)
0 (2 ).2 )) For k = 1 : n do
2
6
6
6
6
6
4
a(0k)
i
(0) =
a(0)
n 1 n
h
1
0
0 0
i
2
3
...
7
"
7
7
7=(
a(nk) 1 75
n(k)
CR (x rn 1 (x)) 0
0 0 1
0
#
xk I ) 6
6
6
6
6
4
a(0k 1)
3
...
7
7
7
7
(
k 1) 7
an 1 5
n(k 1)
where R = fr0(x); :::; rn 1(x); xrn 1 (x)g.
3 ). Compute the entries of VP^ (x) using recursion (3.9).
4 ). Multiply the matrices on the right hand side of (2.7).
In the case of the monomial basis, R = P , this procedure reduces to the conventional
Parker-Forney-Traub algorithm, described in Sec. 1. For polynomials orthogonal on a real
interval, this procedure reduces to the Calvetti-Reichel algorithm [CR93] for inversion of
three-term Vandermonde matrices. In both cases the computational complexity is O(n2) operations, however for general polynomials, satisfying only deg pk (x) = k, the above generalized Parker-Forney-Traub algorithm requires the same amount O(n3) operations as standard
methods. In the next section we show that in another important special case of Szego polynomials our inversion algorithm also admits a simplication, resulting in an order-of-magnitude
reduction in the computational complexity.
4
Inversion of Szeg
o-Vandermonde matrices
As it has been just mentioned, although in the general case the generalized Parker-Forney-Traub algorithm requires O(n3) operations, it has
a favorably low complexity of O(n2) operations for the monomials and for polynomials orthogonal on a real interval. As was just noted, the crucial point that makes this speed-up
possible is the sparsity of the corresponding confederate matrices CR(rn). Clearly, the sparsity of CR(rn) means that the n2 entries of VR(x) are completely dened by a smaller number
O(n) of parameters. Therefore it is not surprising that operating on these parameters we are
able to achieve a faster computation. However in a general situation the confederate matrix
also involves O(n2) parameters, so the generalized algorithms proposed above have the same
complexity O(n3) operations as structure-ignoring algorithms. In the rest of the paper we
consider another important case where R are the Szego polynomials, and show that in this
case our algorithms also allow an eÆcient simplication.
4.2. Recurrence relations for Szeg
o polynomials. Let = f0 (x); :::; n (x)g denote the family of Szego polynomials, i.e., those satisfying the two-term recurrence relations
"
#
" #
1
0 (x)
1 ;
=
#
0 (x)
0 1
"
#
"
#"
#"
#
1
i+1 (x)
i (x)
1
x
0
i+1
=
(4.1)
i+1
1
0 1 #i (x) ;
#i+1 (x)
i+1
4.1. Computational complexity.
9
where the numbers f1 ; 2; :::; n g are called the reection coeÆcients
q (the names Schur
parameters, parcor coeÆcients are also in use), and the numbers k = 1 jk j2 are called
complementary parameters (if jn j = 1 then we set n := 1 for consistency). The Szego
polynomials are orthonormal with respect to a suitable inner product on the unit circle,
< p(x); q (x) >=
Z
p(ei ) [q (ei )] w2()
d
2 ;
and they are of a particular importance in signal processing applications.
In the rest of the paper we shall specialize our algorithms to V(x), which we suggest to
call the Szego-Vandermonde matrices.
To eÆciently specialize the generalized Parker-Forney-Traub algorithm for V(x) we need
to write down the corresponding confederate matrix, for this purposes it will be more convenient to use not (4.1), but recently somewhat ignored three-term recurrence relations for
Szego polynomials (see, e.g., [G48])
1
(x) = 1;
(x) = (x (x) (x));
0
1
0
1
1
0
1 x + k 1 ] (x) k k 1 x (x):
(4.2)
k 2
k
k 1 k k 1
k 1 k
In fact, when Szego rst introduced the polynomials orthogonal on the unit circle, he gave
the expected three-term recursion (4.2). The two-term recursions (4.1) were only recognized
about twenty years later and independently by Szego (1939) and Geronimus (1948).
A straightforward computation easily allows one to convert (4.2) into the n-term recurrence relations of the form (2.9), which describe the corresponding confederate matrix. In
fact, the following statement holds.
k (x) = [
Lemma 4.1
The Szego polynomials in (4.1) (or, equivalently, in (4.2)) satisfy
0 (x) = 1;
1 (x) =
where 0 =
1 (x (x) + (x));
1
0
1 0
0
1, and
1
k (x) = [x k 1(x) + k k 1 k 1(x) + k k 1k 2 k 2 + k k 1 k 2k 3 + :::
k
::: + k k 1 k 2 ::: 2 1 1 (x) + k k 1k 2 ::: 1 0 (x)];
(4.3)
or, equivalently, the confederate matrix (2.11) for b(x) = bn n (x) + ::: + b0 0 (x) is of the
form
2
6
6
6
6
C (b) = 666
6
6
4
1 0
1
0
.
.
.
.
.
.
0
2 1 0
21
2
..
.
3 2 1 0
32 1
3 1
n 1 n 2 :::1 0
n 1 n 2 :::2 1
n 1 n 2 :::3 2
.
.
.
3
..
.
..
0
n 1 n
n 1
.
10
2
n n 1 :::1 0
n n 1 :::2 1
n n 1 :::3 2
b0
bn
b1
bn
b2
bn
n
n
n
3
7
7
7
7
7
.
7
.
.
7
7
7
b
n
2
5
n n 1 n 2
bn n
n n 1 bnbn 1 n
(4.4)
The Hessenberg matrix (4.4) diers from the unitary only in the last column, and this matrix
was written down [G82]) (in a slightly dierent form), and [KP83]. The useful properties
of this unitary Hessenberg matrices were later intensively studied in the signal processing
(where it appears as a system matrix of orthogonal lattice lters), and quite independently
in numerical analysis literature. Below we exploit a simple structure of C(b) to speed-up
the computations with V(x).
4.3. Multiplication of C (n ) by a vector. Recall that in order to specify the
generalized Parker-Forney-Traub algorithm to the Szego-Vandermonde matrices we need to
simplify the computations in the steps 2 and 3 of this algorithm, based on lemmas 2.1 and
2.2, respectively (cf. with the discussion preceding Lemma 2.1). Specically, we need
(a) an eÆcient procedure for multiplication of C (n ) by a vector, used in (3.8) to compute
the coeÆcients of b(x) = Qni=1(x xi ) with respect to the polynomial basis .
(b) an analog of (2.3), i.e., a computationally eÆcient recursion for the successive evaluation
of the generalized Horner polynomials
R^ associated with the R = f0 (x); 1 (x); :::; n 1 (x); b(x)g.
In fact, the rst task in (a) is easily achieved by using the well-known decomposition
C (n ) = H (1 ) H (2 ) ::: H (n 1 ) H~ (n ):
(4.5)
"
#
into the product of the unitary matrices of the form H (k ) = diagfIk 1; k k ; In k 1g
k
k
~
for k = 1; 2; :::; n 1, and Hn = diagfIn 1; ng. Therefore, (4.5) reduces multiplying C(n)
by a vector to n 1 circular rotations, thus suggesting an eÆcient implementation for the
step 2 of the generalized Parker-Forney-Traub algorithm of section 2 for Szego-Vandermonde
matrices.
4.4. Generalized Horner polynomials. To address the second task in (b), we rst
observe that if n = 1 (and, for consistency, n = 1), then the generalized Horner polynomials ^ = f^0(x); :::; ^n(x)g associated with = f0(x); :::; n(x)g are themselves the Szego
polynomials dened by their reection coeÆcients
f^0; ^1 ; :::; ^ng = fn; n 1; :::; 1 ; 0g;
(4.6)
(now it is clear why we adopted earlier the convention 0 = 1). Briey, in this simplest case
the operation of the passage to ^ is reduced to the reversion and complex conjugation for
the reection coeÆcients. However, for R = f0(x); :::; n 1(x); b(x)g the generalized Horner
polynomials R^ are not the Szego polynomials, and moreover, their confederate matrix,
CR^ (b) =
2
6
6
6
6
6
6
4
n 1 n
n
0
...
0
1
bn 1
bn
^0
n 2 n 1 n
n 2n
n
2
bn 2
bn
1
^0
...
0
11
(4.7)
3
1 2 :::n 1 n bbn1 ^0 0 1 :::n 1 n bbn0 ^0
1 2 :::n 2 n 1
0 1 :::n 2 n 1 7
7
7
...
...
7
7
7
5
1 2
0 1 2
1
0 1
is not sparse, which means that we have the n-term recurrence relations for R^. However, the
arguments converse to those used for deducing (4.3) from (4.2) allows to obtain the following
simpler recursion.
Let the polynomial b(x) be arbitrary, and the f0 (x); :::; n(x)g be the rst
n +1 Szego polynomials, specied by f1 ; :::; n g via (4.2), and let 0 = 1. Then the generalized Horner polynomials R^ = fr^0 (x); :::; r^n (x)g associated with R = f0 (x); :::; n 1 (x); b(x)g
are given by
Lemma 4.2
1. Three-term recurrence relations.
1 xr^ (x) ^1^0 r^ (x)g bn 1 ^ :
^1 0
^1 0
bn 0
bn k bn k+1 ^k 1 ^^ 1
^k 1
^k ^k 1
1
r^k (x) = f[ x+
]^r (x) ^ ^ xr^k 2(x)g
^0 r^0 (x):
^k ^k 1 ^k k 1
bn ^k
k 1 k
(4.8)
q
2
where ^k = n k for k = 0; 1; :::; n and ^k = 1 j^k j , ^n = 1 (and if j^n j = 1, then
^n := 1).
r^1 (x) = f
r^0 (x) = 1;
k
k
2. Two-term recurrence relations.
"
#
~0 (x) = 1
~0
^#0 (x)
"
#
~0 bn ;
bn
"
#
~k (x) = 1
^#k (x)
~k
"
1
~k
~k
1
#"
~k 1(x)
x^#k 1 (x) + bn
#
k
;
(4.9)
In a parallel paper [O98] we described the Horner-Szego polynomials and gave an interpretation of the formulas (4.8) of (4.9) using the language of discrete transmission lines (and
used them for eigenvalue computations). Here we use a purely algebraic technique based
on almost-unitary Hessenberg matrices, and apply it to derive fast algorithms similar to the
one of Parker-Forner-Traub.
Either formula (4.8) of (4.9) suggests an eÆcient implementation for the step 3 of the
generalized Parker-Forney-Traub algorithm of section 2 for Szego-Vandermonde matrices.
Summarizing, the overall complexity of the suggested inversion algorithm is O(n2).
5
Conclusion
In this paper we generalized the well-known Parker-Forney-Traub algorithm to polynomialVandermonde matrices. The algorithm is derived by exploiting the properties of the corresponding confederate matrix (i.e., Hessenberg matrix capturing the recurrence relations). In
the important case of Szego polynomials the corresponding Hessenberg matrix diers from
unitary only in the last column. This nice property is exploited to obtain a dramatic simplication of the new algorithms, leading to a favorably small computational complexity O(n2)
operations, as opposed to the usual O(n3) of standard (structure-ignoring) methods.
12
References
[ACR96]
[AGR93]
[B75]
[BKO00]
[BP70]
[C11]
[C55]
[C84]
[F66]
[FF89]
[CR93]
[G48]
[G86]
[G82]
G.Ammar, D.Calvetti and L.Reichel, Continuation methods for the computation
of zeros of Szego polynomials, Linear Algebra and Its Applications, 249 (1996),
125-155.
G.Ammar, W.Gragg and L.Reichel, An analogue for the Szego polynomials of
the Clenshaw algorithm, J. Computational Appl. Math., 46 (1993) pp., 211-216.
S.Barnett, A companion matrix analogue for orthogonal polynomials, Linear
Algebra Appl., 12 (1975), 197-208.
T.Boros, T.Kailath and V.Olshevsky, The Fast Bjorck-Pereyra-type algorithm
for solving Cauchy linear equations, to appear in Linear Algebra and Its Applications, 2000.
Stanford ISL research report, 1995.
A.Bjorck and V.Pereyra, Solution of Vandermonde Systems of Equations, Math.
Comp., 24 (1970), 893-903.
F.Cajori, Horner's method of approximation anticipated by RuÆni, Bull. Amer.
Math. Soc., 17(1911), 301-312.
C.Clenshaw, A note on summation of Chebyshev series, M.T.A.C., 9(51)(1955),
118-120.
T.Constantinescu, On the structure of the Naimark dilation, J. of Operator
Theory, 12: 159-175 (1984).
G.Forney, Concatenated codes, The M.I.T. Press, 1966, Cambridge.
C.Foias and A.E.Frazho, The Commutant Lifting Approach to Interpolation
Problems, Birkhauser Verlag, 1989.
Calvetti, D. and Reichel, L. : Fast inversion of Vandermonde-like matrices involving orthogonal polynomials, BIT, 1993.
L.Y.Geronimus, Polynomials orthogonal on a circle and their applications,
Amer. Math. Translations, 3 p.1-78, 1954 (Russian original 1948).
W.B.Gragg, The QR algorithm for unitary Hessenberg matrices, J.Comput.
Appl. Math., 16 (1986), 1-8.
W.B.Gragg, Positive denite Toeplitz matrices, the Arnoldi process for isometric operators, and Gaussian quadrature on the unit circle (in Russian). In : E.S.
Nikolaev (Ed.), Numerical methods in Linear Algebra, pp. 16-32, Moskow University Press, 1982.
English translation in : J. Comput. and Appl. Math., 46(1993), 183-198.
13
[GL89]
[GO94]
[GO96]
[GS58]
[GR90]
[H1819]
[K69]
[K85]
[K32]
[KO94]
[KP83]
[L1775]
[MB79]
[ML80]
G.Golub and C.Van Loan, Matrix Analysis, second edition, John Hopkins U.
P., Baltimore, 1989.
I.Gohberg and V.Olshevsky, Fast inversion of Chebyshev-Vandermonde matrices, Numerische Mathematik, 67, No. 1 (1994), 71 { 92.
I.Gohberg and V.Olshevsky, A fast generalized Parker-Traub algorithm for inversion of Vandermonde and related matrices, Journal of Complexity, 13(2)
(1997), 208-234.
A short version in pp. in Communications, Computation, Control and Signal
Processing: A tribute to Thomas Kailath, Eds. A.Paulraj, V Roychowdhury and
C.Shaper, Kluwer Academic Publishing, 1996, p.205-221.
U.Grenader and G.Szego, Toeplitz forms and Applications, University of California Press, 1958.
W.B.Gragg and L.Reichel, A divide and conquer method for unitary and orthogonal eigenproblems, Numer. Math., 57 (1990), 695-718.
W.G.Horner, A new method of solving numerical equations of all orders by
continuous approximation, Philos. Trans. Roy. Soc. London, (1819), 308-335.
I.Kaufman, The inversion of the Vandermonde matrix and the transformation
to the Jordan canonical form, IEEE Trans. on Automatic Control, 14(1969),
774 - 777.
H.Kimura, Generalized Schwartz Form and Lattice-Ladder Realizations for Digital Filters, IEEE Transactions on Circuits and Systems, 32, No 11 (1985),
1130-1139.
G. Kowalewski, Interpolation und genaherte Quadratur, Teubner, Berlin, 1932.
T.Kailath and V.Olshevsky, Displacement structure approach to polynomial
Vandermonde and related matrices, preprint, 1994.
T.Kailath and B.Porat, State-space generators for orthogonal polynomials, in Prediction theory and harmonic analysis, The Pesi Masani Volume,
V.Mandrekar and H.Salehi (eds.), pp.131-163, North-Holland Publishing Company, 1983.
J.L.Lagrange, Sur les suites recurrentes, Nouveau Memories de l'Academie
Royale de Berlin, 6(1775), 183-195.
J.Maroulas and S.Barnett, Polynomials with respect to a general basis. I. Theory, J. of Math. Analysis and Appl., 72 : 177 -194 (1979).
M.Morf and D.T.Lee, State-space structure of ladder canonical forms, Proc.
18th Conf. on Control and Design, Dec. 1980, 1221-1224.
14
[O98]
V.Olshevsky, Eigenvector computation for almost unitary Hessenberg matrices
and inversion of Szego-Vandermonde matrices via discrete transmission lines,
Linear Algebra and Its Applications, 285 (1998), 37-67.
[P64]
F.Parker, Inverses of Vandermonde matrices, Amer. Math. Monthly, 71 (1964),
410 - 411.
[R90]
L.Reichel, Newton interpolation at Leja points, BIT, 30(1990), 23 { 41.
[RO91]
L.Reichel and G.Opfer, Chebyshev-Vandermonde systems, Math. of Comp., 57
(1991), 703-721.
[SB80]
J.Stoer and R.Bulirsch, Introduction to Numerical Analysis, Springer-Verlag,
New York, 1980.
[T66]
J. Traub, Associated polynomials and uniform methods for the solution of linear
problems, SIAM Review, 8, No. 3 (1966), 277 { 301.
[TKH83] M.Takizawa, Hisao Kishi and N.Hamada, Synthesis of Lattice Digital Filter
by the State Space Variable Method, Trans. IECE Japan, vol. J65-A (1983),
363-370.
[Wertz65] H.J.Wertz, On the numerical inversion of a recurrent problem : the Vandermonde matrix, IEEE Trans. on Automatic Control, AC-10, 4(1965), 492.
15
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising