Transducer Theory and Streaming Transformations Emmanuel Filiot Universit´

Transducer Theory and Streaming Transformations Emmanuel Filiot Universit´
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Transducer Theory and Streaming
Transformations
Emmanuel Filiot
Université Libre de Bruxelles
1 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Automata
finite string acceptors over a finite alphabet Σ
read-only input tape, left-to-right
finite set of states
Definition (Finite State Automaton)
A finite state automaton (FA) on Σ is a tuple A = (Q, I , F , δ)
where
Q is the set of states,
I ⊆ Q, reps. F ⊆ Q is the set of initial, resp. final, states,
δ : Q × Σ → Q is the transition relation.
L(A) = {w ∈ Σ∗ | there exists an accepting run on w }
2 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Automata – Example
b
b
a
start
q0
q1
a
3 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Automata – Example
b
b
a
q0
start
q1
a
Run on aabaa:
start
q0
a
q1
a
q0
b
q0
a
q1
a
q0
3 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Automata – Example
b
b
a
q0
start
q1
a
Run on aabaa:
start
q0
a
q1
a
q0
b
q0
a
q1
a
q0
L(A) = {w ∈ Σ∗ | w contains an even number of a}
3 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of FA
Expressiveness
FA = regular languages = MSO[+1] = regular expressions = ...
4 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of FA
Expressiveness
FA = regular languages = MSO[+1] = regular expressions = ...
Closure Properties
closed under Boolean operations (union, intersection,
complement).
closed under various extensions:
non-determinism (NFA): δ ⊆ Q × Σ × Q
two-way input head (2NFA): δ ⊆ Q × Σ × {−1, 0, 1} × Q
regular look-ahead: δ ⊆ Q × Σ × Reg × Q
alternation: δ : Q × Σ → B(Q) (Boolean formulas over Q)
4 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of FA
Expressiveness
FA = regular languages = MSO[+1] = regular expressions = ...
Closure Properties
closed under Boolean operations (union, intersection,
complement).
closed under various extensions:
non-determinism (NFA): δ ⊆ Q × Σ × Q
two-way input head (2NFA): δ ⊆ Q × Σ × {−1, 0, 1} × Q
regular look-ahead: δ ⊆ Q × Σ × Reg × Q
alternation: δ : Q × Σ → B(Q) (Boolean formulas over Q)
Decision Problems
Membership, emptiness, universality, inclusion, equivalence ... are
decidable.
4 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
From Languages to Transductions
Let Σ and ∆ be two finite alphabets.
Definition
Language on Σ
Transduction from Σ to ∆
function from Σ∗ to {0, 1}
relation R ⊆ Σ∗ × ∆∗
defined by automata
defined by transducers
accept strings
transform strings
transducer = automaton + output mechanism.
5 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers
6 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers
read-only left-to-right input head
write-only left-to-right output head
finite set of states
7 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers
read-only left-to-right input head
write-only left-to-right output head
finite set of states
Definition (Finite State Transducers)
A finite state transducer from Σ to ∆ is a pair T = (A, O) where
A = (Q, I , F , δ) is the underlying automaton
O is an output morphism from δ to ∆∗ .
a
If t = q →
− q 0 ∈ δ, then O(t) defines its output.
a|w
q −−→ q 0 denotes a transition whose output is w ∈ ∆∗ .
7 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers
read-only left-to-right input head
write-only left-to-right output head
finite set of states
Definition (Finite State Transducers)
A finite state transducer from Σ to ∆ is a pair T = (A, O) where
A = (Q, I , F , δ) is the underlying automaton
O is an output morphism from δ to ∆∗ .
a
If t = q →
− q 0 ∈ δ, then O(t) defines its output.
a|w
q −−→ q 0 denotes a transition whose output is w ∈ ∆∗ .
Two classes of transducers:
DFT if A is deterministic
NFT if A is non-deterministic.
7 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Some applications
language and speech processing (e.g. see work by Mehryar
Mohri)
model-checking infinite state-space systems1
verification of web sanitizers2
string pattern matching
1
A survey of regular model checking, P. Abdulla, B. Jonsson, M. Nilsson, M.
Saksena. 2004
2
see BEK, developped at Microsoft Research
8 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 1
b|
b|
a|a
start
q0
q1
a|a
9 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 1
b|
b|
a|a
q0
start
q1
a|a
Run on aabaa:
start
q0
a|a
q1
a|a
q0
b|
q0
a|a
q1
a|a
q0
T (aabaa)=a.a..a.a=aaaa.
9 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 1
b|
b|
a|a
start
q0
q1
a|a
10 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 1
b|
b|
a|a
q0
start
q1
a|a
Run on aaba:
start
q0
a|a
q1
a|a
q0
b|
q0
a|a
q1
T (aaba)= undefined
10 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 1
b|
b|
a|a
start
q0
q1
a|a
Semantics
dom(T ) = {w ∈ Σ∗ | #a w is even}
R(T ) = {(w , a#a w ) | w ∈ dom(T )}
11 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 2
= white space
|
a|a
|
start
q0
q1
a|a
12 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 2
= white space
|
a|a
|
start
q0
q1
a|a
Semantics
Replace blocks of consecutive white spaces by a single white space.
T ( aa
a ) = aa a
12 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 3
= white space
|
a|a
|
|
q2
|
q0
q1
a|a
13 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 3
= white space
|
a|a
|
|
q2
|
q0
q1
a|a
Semantics
Replace blocks of consecutive white spaces by a single white space
and
remove the last white spaces (if any).
T ( aa
a ) = aa a
13 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Example 3
= white space
|
a|a
|
|
q2
|
q0
q1
a|a
Semantics
Replace blocks of consecutive white spaces by a single white space
and
remove the last white spaces (if any).
T ( aa
a ) = aa a
Non-deterministic but still defines a function: functional NFT
13 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Is non-determinism needed ?
|
a|a
|
|
q2
|
q0
q1
a|a
14 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Is non-determinism needed ?
|
a|a
|
|
q2
|
q0
q1
a|a
≡
|
a|a
|
q3
q4
a| a
14 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
How to get a deterministic FT ?
|
a|a
|
|
q2
|
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
[Conclusion]
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
q0
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
q0
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
q0
|
q1
, q2
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
q0
|
q1
, q2
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
q0
|
q1 ( ), q2 ()
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
q0
|
|
q1 ( ), q2 ()
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to get a deterministic FT ?
|
a|a
|
|
|
q2
q0
q1
a|a
extend automata subset construction with outputs
output the longest common prefix
a|a
|
q0
|
q1 ( ), q2 ()
a| a
15 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Can we always get an equivalent deterministic FT ?
16 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Can we always get an equivalent deterministic FT ?
not in general: DFT define functions, NFT define relations
what about functional NFT ?
16 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Can we always get an equivalent deterministic FT ?
not in general: DFT define functions, NFT define relations
what about functional NFT ?
a|c
q4
c|c
q2
a|b
a|c
q0
a|b
q1
b|b
q3
initial
Semantics

 n
a b 7→ b n+1
R(T ) :
 n
a c 7→ c n+1
functional but not determinizable
16 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
q2
a|b
a|c
q0
a|b
q1
b|b
q3
initial
Subset construction:
q0
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
a|b
a|c
q2
q0
a|b
q1
b|b
q3
initial
Subset construction:
q0
a|
q1
q2
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
a|b
a|c
q2
q0
a|b
q1
b|b
q3
initial
Subset construction:
q0
a|
q1 (b)
q2 (c)
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
a|b
a|c
q2
a|b
q0
q1
b|b
q3
initial
Subset construction:
q0
a|
q1 (b)
q2 (c)
a|
q1 (bb)
q2 (cc)
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
a|b
a|c
q2
a|b
q0
q1
b|b
q3
initial
Subset construction:
q0
a|
q1 (b)
q2 (c)
a|
q1 (bb)
q2 (cc)
a|
q1 (bbb)
q2 (ccc)
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Subset construction fails ...
a|c
c|c
q4
a|b
a|c
q2
a|b
q0
q1
b|b
q3
initial
Subset construction:
q0
a|
q1 (b)
q2 (c)
a|
q1 (bb)
q2 (cc)
a|
q1 (bbb)
......
q2 (ccc)
17 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to guarantee termination of subset construction?
LAG
LAG (u, v ) = (u 0 , v 0 ) such that u = `u 0 , v = `v 0 and ` = lcp(u, v ).
E.g. LAG (abbc, abc) = (bc, c).
18 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How to guarantee termination of subset construction?
LAG
LAG (u, v ) = (u 0 , v 0 ) such that u = `u 0 , v = `v 0 and ` = lcp(u, v ).
E.g. LAG (abbc, abc) = (bc, c).
Lemma (Twinning Property)
Subset construction terminates iff for all such situations
u1 |v1
q
u1 |w1
p
u2 |v2
q
q0
p
u2 |w2
it is the case that LAG (v1 , w1 ) = LAG (v1 v2 , w1 w2 ).
18 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Determinizability is decidable
Theorem (Choffrut 77, Beal Carton Prieur Sakarovitch 03)
Given a functional NFT T , the following are equivalent:
1
it is determinizable
2
the twinning property holds.
Moreover, the twinning property is decidable in PTime.
Proof.
Intuition
If TP holds, then subset construction terminates and produces
an equivalent DFT
for the converse, uses the fact that TP is
machine-independent: for all T ≡ T 0 , T |= TP iff T 0 |= TP.
19 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Determinizability is decidable
Theorem (Choffrut 77, Beal Carton Prieur Sakarovitch 03)
Given a functional NFT T , the following are equivalent:
1
it is determinizable
2
the twinning property holds.
Moreover, the twinning property is decidable in PTime.
Proof.
Intuition
If TP holds, then subset construction terminates and produces
an equivalent DFT
for the converse, uses the fact that TP is
machine-independent: for all T ≡ T 0 , T |= TP iff T 0 |= TP.
Almost true ...
19 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
True if ...
a|a
q0
b
a|ab
q1
q0 ()
a|a
q0 ()
a|a
q1 (b)
c|c
c|c
q2
q1 ()
subsequential transducers are deterministic but can output a
string in each accepting states
in the previous theorem: “determinizable” ↔ “there exists an
equivalent subsequential transducer”
subsequential transducers ≡ DFT if last string symbol is
unique
20 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Application: analysis of streaming transformations
Bounded Memory Problem
Hypothesis:
input string is received as a (very long) stream
output string is produced as a stream
Input: a transformation defined by some functional NFT
Output: can I realize this transformation with bounded memory ?
∃B ∈ N · ∀u ∈ dom(T )
T (u) can be computed with B-bounded memory ?
21 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming Model
Deterministic Turing Transducer
22 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Bounded Memory Problem – Examples
T1 :
T2 :


an b 7→ b n+1

an c
a
b
7→
Not bounded memory
c n+1
7→ a b
Bounded memory
23 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Bounded Memory Problem – Examples
T1 :
T2 :


an b 7→ b n+1

an c
a
b
7→
Not bounded memory
c n+1
7→ a b
Bounded memory
Theorem
For all functional NFT T , the following are equivalent:
1
T is bounded memory
2
T is determinizable
3
T satisfies the twinning property.
Proof based on the following two observations:
1 any DFT is bounded memory
2 bounded memory Turing Transducer ≡ DFT
23 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Closure Properties of Finite State Transducers
Domain, co-domain
The domains and co-domains of NFT are regular.
T −1
T
T1 ∪ T2
T1 ∩ T2
T1 ◦ T2
NFT
no
no
yes
no
yes
DFT
no
no
no
no
yes
Table: Closure Properties for NFT and DFT.
24 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Closure Properties of Finite State Transducers
Domain, co-domain
The domains and co-domains of NFT are regular.
T −1
T
T1 ∪ T2
T1 ∩ T2
T1 ◦ T2
NFT
no
no
yes
no
yes
DFT
no
no
no
no
yes
Table: Closure Properties for NFT and DFT.
Non-closure by intersection
1
R(T1 ) = {(am b n , c m ) | m, n ≥ 0}
2
R(T2 ) = {(am b n , c n ) | m, n ≥ 0}
3
R(T1 ) ∩ R(T2 ) = {(an b n , c n ) | n ≥ 0}
24 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Decision problems
Membership (u, v ) ∈ R(T )?
Emptiness R(T ) = ∅?
Type checking T (Lin ) ⊆ Lout ?
Equivalence R(T1 ) = R(T2 )?
Inclusion R(T1 ) ⊆ R(T2 )?
emptiness /
type checking
equiv /
membership
(vs NFA)
inclusion
NFT
PTime
PSpace-c
undec
DFT
PTime
PSpace-c
PTime
Table: Decision problems for NFT and DFT.
Undecidability of equivalence and inclusion proved in [Griffiths68].
25 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Functional Finite State Transducers
A transduction (transducer) is functional if each word has at most
1 image.
Theorem (Gurari and Ibarra 83)
Functionality is decidable in PTime for NFT.
Theorem
The equivalence and inclusion of functional NFT is PSpace-c.
Proof.
T1 is included in T2 if and only if
dom(T1 ) ⊆ dom(T2 ), and
T1 ∪ T2 is functional.
26 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
k-valued Finite State Transducers
A transduction (transducer) is k-valued if each word has at most k
images.
Theorem ( GI83, Web89, SdS08 )
Let k ∈ N be fixed.
k-valuedness is decidable in PTime for NFT.
Theorem (IK86, Web88)
The equivalence and inclusion of k-valued NFT are PSpace-c.
27 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Extensions of NFT
28 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Extensions of NFT
Various more expressive extensions have been considered:
1
two-way input tape
2
string variables (Alur Cerny 2010)
3
pushdown stack
29 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
`|
2
3
Output Tape
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
Output Tape
`|
2
3
d
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
1
Output Tape
`|
2
d
3
e
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
3
s
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
3
s
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
s
3
e
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
s
e
3
r
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
`
s
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
s
e
3
r
t
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
s
e
3
r
t
s
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers (2NFT)
Input Tape
s
`
t
r
e
s
s
e
d
a
head
α|α, −1
α|, +1
a|, −1
`|
1
Output Tape
2
d
e
s
s
e
3
r
t
s
head
30 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Two-way finite state transducers – Properties
Main Properties of 2NFT
1
still closed under composition (Chytil Jakl 77)
2
equivalence of functional 2NFT is decidable (Culik,
Karhumaki, 87)
3
functional 2NFT ≡ 2DFT (Hoogeboom Engelfriet 01, De
Souza 13)
Logical Characterization (Hoogeboom Engelfriet 01)
2DFT ≡ MSO transductions
2DFT define regular functions.
31 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
MSO Transductions (Courcelle)
input string seen as the logical structure over
{succ, (laba )a∈Σ }
output predicates defined with MSO formulas interpreted over
the input structure
32 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
MSO Transductions (Courcelle)
input string seen as the logical structure over
{succ, (laba )a∈Σ }
output predicates defined with MSO formulas interpreted over
the input structure
succ
s
succ
t
succ
r
succ
e
succ
s
succ
s
succ
e
d
32 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
MSO Transductions (Courcelle)
input string seen as the logical structure over
{succ, (laba )a∈Σ }
output predicates defined with MSO formulas interpreted over
the input structure
succ
s
succ
t
succ
r
succ
e
succ
s
succ
s
succ
e
d
φsucc (x, y ) ≡ succ(y , x)
φlaba (x) ≡ laba (x)
32 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
MSO Transductions (Courcelle)
input string seen as the logical structure over
{succ, (laba )a∈Σ }
output predicates defined with MSO formulas interpreted over
the input structure
succ
s
succ
r
t
succ
succ
succ
succ
e
succ
succ
s
succ
succ
s
succ
succ
e
succ
d
succ
φsucc (x, y ) ≡ succ(y , x)
φlaba (x) ≡ laba (x)
32 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
MSO Transductions (Courcelle)
input string seen as the logical structure over
{succ, (laba )a∈Σ }
output predicates defined with MSO formulas interpreted over
the input structure
s
r
t
succ
succ
e
succ
s
succ
s
succ
e
succ
d
succ
φsucc (x, y ) ≡ succ(y , x)
φlaba (x) ≡ laba (x)
32 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming String Transducers (Alur, Cerny, 2010)
On every transitions, a finite set of variables can be updated by
appending a string: x := x.u
prepending a string: x := u.x
concatenating two variables: x := yz
33 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming String Transducers (Alur, Cerny, 2010)
On every transitions, a finite set of variables can be updated by
appending a string: x := x.u
prepending a string: x := u.x
concatenating two variables: x := yz
α|x := α.x
q0
x
R(T ) = mirror
33 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming String Transducers (Alur, Cerny, 2010)
On every transitions, a finite set of variables can be updated by
appending a string: x := x.u
prepending a string: x := u.x
concatenating two variables: x := yz
x := x.b
a y := y .c
α|x := α.x
x .b
b|x :=
q0
x
x
q2
y
q0
c |y :=
y .c
R(T ) = mirror
q1
R(T ) = an α 7→ αn+1
33 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streaming String Transducers
Theorem (Alur Cerny 2010)
The following models are expressively equivalent:
1
two-way DFT
2
MSO transductions
3
deterministic (one-way) streaming string transducers with
copyless update
Moreover, SSTs have good algorithmic properties and have been
used to analyse list processing programs (Alur Cerny 2011).
34 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Pushdown Transducers
Definition
A pushdown transducer is a pair (A, O) where A is a pushdown
automaton and O is an output morphism.
(Bad) Properties
closure under composition is lost
Functionality, determinizability, equivalence and inclusion of
functional transducers are lost.
35 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
DFTs
fNFTs
NFTs
2DFTs
f2NFTs
2NFTs
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
` u a 7→ mirror(u)
DFTs
fNFTs
NFTs
⊂
⊂
⊂
2DFTs
f2NFTs
2NFTs
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
{(a, a), (a, b)}
DFTs
fNFTs
⊂
f2NFTs
NFTs
⊂
2NFTs
⊂
⊂
2DFTs
⊂
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
an α 7→ αn+1
DFTs
⊂
fNFTs
⊂
f2NFTs
NFTs
⊂
2NFTs
⊂
⊂
2DFTs
⊂
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
DFTs
≡
f2NFTs
NFTs
⊂
2NFTs
⊂
2DFTs
⊂
⊂
fNFTs
⊂
⊂
[ De Souza (13)]
≡MSOT [ Engelfriet,Hoogeboom (01)]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
PTIME
[Choffrut (77)]
[Weber, Klemm (95)]
[Beal,Carton,Prieur,Sakarovitch(03)]
?
DFTs
≡
f2NFTs
NFTs
⊂
2NFTs
⊂
2DFTs
⊂
⊂
fNFTs
⊂
⊂
[ De Souza (13)]
≡MSOT [ Engelfriet,Hoogeboom (01)]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
PTIME
PTIME
[Choffrut (77)]
[Schützenberger (75)]
[Weber, Klemm (95)]
[Gurari, Ibarra (83)]
[Beal,Carton,Prieur,Sakarovitch(03)]
[Beal,Carton,Prieur,Sakarovitch(03)]
?
DFTs
≡
f2NFTs
⊂
NFTs
⊂
2NFTs
⊂
fNFTs
⊂
⊂
⊂
2DFTs
?
[ De Souza (13)]
≡MSOT [ Engelfriet,Hoogeboom (01)]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
PTIME
PTIME
[Choffrut (77)]
[Schützenberger (75)]
[Weber, Klemm (95)]
[Gurari, Ibarra (83)]
[Beal,Carton,Prieur,Sakarovitch(03)]
[Beal,Carton,Prieur,Sakarovitch(03)]
?
DFTs
≡
f2NFTs
⊂
NFTs
⊂
2NFTs
⊂
fNFTs
⊂
⊂
⊂
2DFTs
?
?
[ De Souza (13)]
decidable
≡MSOT [ Engelfriet,Hoogeboom (01)]
[Culik,Karhumaki (87) ]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
PTIME
PTIME
[Choffrut (77)]
[Schützenberger (75)]
[Weber, Klemm (95)]
[Gurari, Ibarra (83)]
[Beal,Carton,Prieur,Sakarovitch(03)]
[Beal,Carton,Prieur,Sakarovitch(03)]
?
DFTs
?
⊂
fNFTs
⊂
2NFTs
⊂
NFTs
⊂
⊂
?
⊂
[Filiot,Gauwin,Reynier,Servais (13)]
2DFTs
≡
f2NFTs
?
[ De Souza (13)]
decidable
≡MSOT [ Engelfriet,Hoogeboom (01)]
[Culik,Karhumaki (87) ]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Finite State Transducers – Summary
D=”(input) deterministic”
f=”functional”
PTIME
PTIME
[Choffrut (77)]
[Schützenberger (75)]
[Weber, Klemm (95)]
[Gurari, Ibarra (83)]
[Beal,Carton,Prieur,Sakarovitch(03)]
[Beal,Carton,Prieur,Sakarovitch(03)]
?
DFTs
?
⊂
⊂
fNFTs
⊂
⊂
⊂
?
NFTs
[Filiot,Gauwin,Reynier,Servais (13)]
2DFTs
≡
f2NFTs
⊂
? open
2NFTs
?
[ De Souza (13)]
decidable
≡MSOT [ Engelfriet,Hoogeboom (01)]
[Culik,Karhumaki (87) ]
≡ Streaming String Transducers [ Alur, Černý, 2010]
36 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
A word about infinite strings
most transducer models can be extended to (right-) infinite
strings
Büchi / Muller accepting conditions
most of the results seen so far still hold with some
complications ...
determinization of one-way transducers: TP is too strong
a|aa
start
q0
a|a
start
q1
deterministic 2way < functional 2way:

 ω
a if infinite number of ’a’
T : u 7→

u otherwise
functional 2way ≡ determinitic 2way + ω-regular look-ahead
≡ ω-MSO transductions ≡ ω-SST (Alur,Filiot,Trivedi,12)
37 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Transducers for Nested Words (∼ Trees)
38 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Motivations
Streaming XML Transformations
XML are words with a nesting structure
XML documents can be (very) wide but usually not deep
in a streaming setting, not reasonable to keep the entire
document in memory
bounded memory streaming transformations ?
39 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Motivations
Streaming XML Transformations
XML are words with a nesting structure
XML documents can be (very) wide but usually not deep
in a streaming setting, not reasonable to keep the entire
document in memory
bounded memory streaming transformations ?
Visibly Pushdown Transducers (VPTs)
extend Visibly Pushdown Automata (Alur Madhusudan 04)
well-suited for streaming nested words transformations
bounded memory analysis for VPT transductions.
39 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Structured Alphabet
Definition (Structured Alphabet)
A structured alphabet, Σ, is a set Σ = Σc ] Σi ] Σr , where
Σc are call symbols,
Σi are internal symbols,
Σr , are return symbols.
a nested word is a word over a structured alphabet
c1 c2 a r1
it is well-nested if there is no pending call nor return symbols
c1 c2 a r2 b r1
40 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Nested Words vs Trees
Encoding
Well-nested words ≡ linearizations of trees
nested words are well-suited to model tree streams
41 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Visibly Pushdown Automata (VPAs) [Alur,Madhusudan,04]
VPAs = Pushdown Automata on structured alphabet
Σ = Σc ] Σr ] Σi :
push one stack symbol on call symbols Σc
pop one stack symbol on return symbols Σr
don’t touch the stack on internal symbols Σi
in this talk, accept on empty stack and final state
42 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Visibly Pushdown Automata (VPAs) [Alur,Madhusudan,04]
VPAs = Pushdown Automata on structured alphabet
Σ = Σc ] Σr ] Σi :
push one stack symbol on call symbols Σc
pop one stack symbol on return symbols Σr
don’t touch the stack on internal symbols Σi
in this talk, accept on empty stack and final state
r , −γ
q3
b
q2
c, +γ
i
q1
c, +γ
q0
r , −γ
c, +γ
initial
c, +γ
q4
i
q5
a
q6
L(A) = {c n i r n a | n > 0} ∪ {c n i r n b | n > 0}
42 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of VPA
NFA < VPA < PA
close under all Boolean operations
NFA algorithmic properties are preserved (equivalence,
universality, ...)
applicatons in
computer-aided verification
XML processing
see http://www.cs.uiuc.edu/∼madhu/vpa/
43 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Visibly Pushdown Transducers (VPTs)
Definition
Pair (A, O) where A : VPA and O is an output morphism.
r |b, −γ
q3
b|
q2
c|b, +γ
i|
q1
c|b, +γ
initial
q0
r |a, −γ
c|a, +γ
c|a, +γ
q4
i|
q5
a|
q6
R(T ) = {(c n i r n a, a2n ) | n > 0} ∪ {(c n i r n b, b 2n ) | n > 0}
44 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of Visibly Pushdown Transducers
NFT < VPT < PT
dVPTs < (functional) VPT
closed under composition if the output is well-nested
closed under VPA-lookahead
functionality is decidable in PTime
k-valuedness is decidable
equivalence of functional VPTs is decidable (in PTime of
dVPTs)
decidable typechecking problem (if the output is well-nested)
45 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Properties of Visibly Pushdown Transducers
NFT < VPT < PT
dVPTs < (functional) VPT
closed under composition if the output is well-nested
closed under VPA-lookahead
functionality is decidable in PTime
k-valuedness is decidable
equivalence of functional VPTs is decidable (in PTime of
dVPTs)
decidable typechecking problem (if the output is well-nested)
Open Problems: equivalence of k-valued VPTs,
determinizability
more details in F. Servais’s Phd thesis
45 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Why is determinizability more difficult?
r |aa, −γ
q2
c|, +γ
i|
q1
initial
c|, +γ
q0
r |a, −γ
c|a, +γ
c|a, +γ
p1
i|
p2
46 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Why is determinizability more difficult?
r |aa, −γ
q2
c|, +γ
i|
q1
c|, +γ
q0
r |a, −γ
c|a, +γ
initial
c|a, +γ
p1
i|
p2
It is determinizable by:
initial
q0
r |a, −γ
c|a, +γ
c|a, +γ
p1
i|
p2
but lag increase arbitrarily in (p1 , q1 ).
46 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streamability Problem [F, Gauwin, Reynier, Servais, 11]
Streaming evaluation: avoid the storage of the whole input
Fix a functional (non-deterministic) VPT T.
How much memory is needed to compute T(u) from an input
stream u?
47 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streamability Problem [F, Gauwin, Reynier, Servais, 11]
Streaming evaluation: avoid the storage of the whole input
Fix a functional (non-deterministic) VPT T.
How much memory is needed to compute T(u) from an input
stream u?
47 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streamability Problem [F, Gauwin, Reynier, Servais, 11]
Streaming evaluation: avoid the storage of the whole input
Fix a functional (non-deterministic) VPT T.
How much memory is needed to compute T(u) from an input
stream u?
Streamability Problem
Given a VPT T, decide if T defines a
transformation that can be evaluated with
memory O(f (height(u))?
47 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Streamability Problem [F, Gauwin, Reynier, Servais, 11]
Streaming evaluation: avoid the storage of the whole input
Fix a functional (non-deterministic) VPT T.
How much memory is needed to compute T(u) from an input
stream u?
Streamability Problem
Given a VPT T, decide if T defines a
transformation that can be evaluated with
memory O(f (height(u))?
Decidable in NP for VPTs
47 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Determinizability is too strong
Obs: Deterministic VPTs are always streamable (no output lag)
48 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Determinizability is too strong
Obs: Deterministic VPTs are always streamable (no output lag)
However: determinizable VPTs < streamable VPTs:
R(T ) : c n i r n α 7→ α2n
r |b, −γ
q3
b|
q2
c|b, +γ
i|
q1
c|b, +γ
initial
q0
n>0
r |a, −γ
c|a, +γ
c|a, +γ
q4
i|
q5
a|
q6
Streamable but not determinizable !
48 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
Definition
For all such situations
u1 |v1
q
u1 |w1
p
u2 |v2
q
q0
p
u2 |w2
it is the case that LAG (v1 , w1 ) = LAG (v1 v2 , w1 w2 ).
49 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
Definition
For all such situations
u1 |v1
(q, σ)
u1 |w1
(p, σ 0 )
u2 |v2
(q, σ)
(q0 , ⊥)
u2 |w2
(p, σ 0 )
it is the case that LAG (v1 , w1 ) = LAG (v1 v2 , w1 w2 ).
49 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
50 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
50 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
50 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
50 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
Theorem
Given a functional VPT T , T is streamable iff the twinning
property holds.
It can be decided in NPtime.
51 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Twinning Property for VPTs
Theorem
Given a functional VPT T , T is streamable iff the twinning
property holds.
It can be decided in NPtime.
TP is machine-independent: streamable VPTs is class of
transductions.
decidability based on reversal-bounded pushdown counter
machines
same result extend to strongly streamable (memory depends
only on current height)
51 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Other tree transducer models
top-down tree transducers
q(f (x1 , . . . , xn )) → C [q1 (xi1 ), . . . , qp (xip )]
(see TATA3 book
macro tree transducers
fun q(t1 t2 t3 t4 t)=
if t = a() then
return F (t1,t2)
else
if t=g(u,v) then
return C(q’(t1,t2,u), q’’(t3,t4,v))
see Joost Engelfriet and Sebastian Maneth’s work
3
Tree Automata Techniques and Applications, tata.gforge.inria.fr
52 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Problem
53 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Problem (aka Church Synthesis Problem)
Definition (Church 57)
R a relation, or requirements, from a domain D to a domain
D0
synthesize a program P such for all X ∈ D, (X , P(X )) ∈ R.
54 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Problem (aka Church Synthesis Problem)
Definition (Church 57)
R a relation, or requirements, from a domain D to a domain
D0
synthesize a program P such for all X ∈ D, (X , P(X )) ∈ R.
Reactive System Synthesis
Let Σin and Σout be to finite alphabets.
reactive systems continuously react to stimuli produced by
some uncontrollable environment
D = Σωin , D 0 = Σωout
R is a synchronous relation given by a (non-deterministic)
symbol-to-symbol Büchi transducer
P is a Mealy machine (deterministic symbol-to-symbol
transducer)
54 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Reactive System Synthesis: Example
Σin = {req, nop}
Σout = {grant, nop}.
Requirement R: if there is a request, it must be eventually
granted
nop|nop
|nop
req|nop
q0
q21
|grant
req|grant
55 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Reactive System Synthesis: Example
Σin = {req, nop}
Σout = {grant, nop}.
Requirement R: if there is a request, it must be eventually
granted
nop|nop
|nop
req|nop
q0
q21
|grant
req|grant
Possible programs (Mealy machines) that realize R:
nop|nop
nop|nop
req|nop
q0
q0
q1
|grant
req|grant
55 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
:
Player out (Σout ) :
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
Player out (Σout ) :
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
Player out (Σout ) : o1
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
Player out (Σout ) : o1
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
Player out (Σout ) : o1 o2
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
Player out (Σout ) : o1 o2
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
Player out (Σout ) : o1 o2 o3
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
Player out (Σout ) : o1 o2 o3
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
Player out (Σout ) : o1 o2 o3 o4
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
i5
Player out (Σout ) : o1 o2 o3 o4
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
i5
Player out (Σout ) : o1 o2 o3 o4 o5
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
i5
...
Player out (Σout ) : o1 o2 o3 o4 o5
...
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
i5
...
Player out (Σout ) : o1 o2 o3 o4 o5
...
Def: Player out wins if (i1 i2 i3 . . ., o1 o2 o3 . . . ) ∈ R.
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Church Game
Definition
turn-based game between two players
Player in chooses input symbols in Σin
Player out chooses output symbols in Σout
they play during an infinite number of rounds.
Player in (Σin )
: i1
i2
i3
i4
i5
...
Player out (Σout ) : o1 o2 o3 o4 o5
...
Def: Player out wins if (i1 i2 i3 . . ., o1 o2 o3 . . . ) ∈ R.
Prop: There exists a program that realizes the requirements
R iff Player out has a winning strategy.
56 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
State of the Art
reactive system synthesis from ω-regular specifications is
decidable (Büchi Landweber 69)
reactive system synthesis from LTL specifications is
2-ExpTime-c (Pnueli Rosner 89)
several tools for LTL synthesis:
Lily (Jobstmann Bloem 06)
Acacia (Filiot Jin Raskin 09)
Unbeast (Ehlers 10)
very active community in game theory for synthesis
quantitative games
multi-player games
stochastic games
...
57 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Acacia Tool
http://lit2.ulb.ac.be/acaciaplus
58 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
How is it related to transducer theory?
reactive systems are streaming machines
from a relation R, extract a function f such that:
1
2
3
dom(R) ⊆ dom(f )
for all u ∈ dom(R), f (u) ∈ R(u).
f is a deterministic symbol-to-symbol transducer
this problem is known as the uniformization problem in
transducer theory
equivalently, is there a bounded memory (symbol-to-symbol)
function f such that f ⊆ R and dom(R) ⊆ dom(f ) ?
59 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Conclusion
60 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Contributions
finite transducers have good closure and algorithmic properties
nicely extend to visibly pushdown transducers
streamability problem ≡ synthesis problem
61 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Open Problems and Future Work
Open problems
equivalence of k-valued VPTs
determinizability of VPTs
extension of streaming results to more expressive transducers,
e.g. macro tree transducers
shift from reactive systems to list processing program synthesis
62 / 63
[Finite State Transducers]
[Extensions of NFT]
VPTs
[Church Problem]
[Conclusion]
Publications
E. Filiot, O. Gauwin, P.-A. Reynier, F. Servais: From Two-Way to
One-Way Finite State Transducers, LICS 2013.
R. Alur, E. Filiot, A. Trivedi: Regular Transformations of Infinite Strings,
LICS 2012.
E. Filiot and F. Servais: Visibly Pushdown Transducers with Look Ahead,
SOFSEM 2012.
E. Filiot, O. Gauwin, P.-A. Reynier and F. Servais: Streamability of
Nested Word Transductions, FSTTCS 2011.
E. Filiot, N. Jin, J.-F. Raskin: Antichain and Compositional Algorithms
for LTL Synthesis, FMSD 2011.
E. Filiot, J.-F. Raskin, P.-A. Reynier, F. Servais and J.-M. Talbot:
Properties of Visibly Pushdown Transducers, MFCS 2010.
J.-F. Raskin, F. Servais: Visibly Pushdown Transducers, ICALP 2008.
63 / 63
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement