USOO5675752A
United States Patent (19)
11
45
Scott et al.
Patent Number:
Date of Patent:
5,544,305 8/1996 Ohmaye et al. ........................ 395/161
5,555,369 9/1996 Menendez et al. ..................... 395/161
54 INTERACTIVE APPLICATIONS
GENERATOR FOR AN INTERACTIVE
PRESENTATION ENVIRONMENT
FOREIGN PATENT DOCUMENTS
0513553 11/1992 European Pat. Off. .
75 Inventors: Edward W. Scott, Anaheim Hills;
Richard Sagey, Laguna Niguel, both of
OTHER PUBLICATIONS
Calif.; Toshiharu Fukui,
Kanagawa-ken, Japan; Hussein F.
The Visual Computer, "Environment for rapidly creating
interactive design tools”. Brad A. Myers and Brad Vander
Zanden, vol. 8, No. 2, pp. 94-116, Feb. 1992.
Akasheh, Costa Mesa, Calif.; Kunjan
Zaveri, Arcadia, Calif.; Marc Booth,
La Habra, Calif.
Primary Examiner Raymond J. Bayerl
Attorney, Agent, or Firm-Blakely, Sokoloff, Taylor &
73 Assignees: Sony Corporation, Tokyo, Japan; Sony
Zafiman
Trans Com Inc., Irvine, Calif.
ABSTRACT
57
21
22
51
52
Appl. No.: 307,941
Filed:
An interactive applications generator is utilized to generate
an interactive application environment for use on a single
server multi-client network computer system. The interac
tive applications generator permits a user to define a graphi
Sep. 15, 1994
Int. C. ... G06F 3/14
U.S. C. ......................... 395/333; 395/334; 395/335;
cal user interface (GUI), that contains one or more display
395/356; 395/972; 395/328; 395/967; 395/807
58
Field of Search ..................................... 395/155, 161,
395/154, 160, 157, 159, 156, 147, 148,
145, 700, 333-335,327,328,972,967,
342, 353,356,357, 806, 807, 762, 776,
777, 779, 782, 792,936, 942,943, 704,
710, 712,326,340,348, 352, 701
References Cited
56
U.S. PATENT DOCUMENTS
5,191,645
5,317,732
5,363,482
5,428,731
5,430,872
5,500.936
5,519,828
5,539,871
5544,302
7/1995 Dahod et al.
3/1996 Allen et al. ..
... 395,159
395/159 X
395/157
... 395,154
... 395/600
... 395/156
5/1996 Rayner ...
... 395/161
7/1996 Gibson ...
... 39.5/154
3/1993 Carlucci et al. .....
5/1994 Gerlach, Jr. et al.
11/1994 Victor et al. .....
6/1995 Powers, III .....
screens, for a customized interactive application environ
ment. The interactive applications generator contains a
screen template editor, a media frame editor, a screen editor,
and an interactive presentation editor. The screen template
editor is utilized to create generic screen templates, that
contain generic screen template elements. The media frame
editor is utilized to generate media frames by combining
multimedia component elements in a time synchronized
manner. The screen editor is utilized to generate the display
screens including assigning functionality. In order to create
a unique instance for each display screen for a particular
interactive applications environment, the screen editor ref
erences generic screen templates and media frames. The
interactive presentation editor is utilized to create the hier
archical structures that defines presentation of the display
screens for the particular interactive presentation environ
ment.
64 Claims, 52 Drawing Sheets
8/1996 Nguyen ................................... 395/6
nteractive
Presentation
Editor
200
Output
Generator
Screer
C
Screen
emplate Editor
C E.O
--
21
(E)
O
image
Editor(s)
255
5,675,752
Oct. 7, 1997
Editor
Text
230
Editor(s)
Graphical
Editor(s)
Audio
Editor(s)
260
Video
Editor(s)
270
Animation
Editor(s)
U.S. Patent
Oct. 7, 1997
Sheet 1 of 52
5,675,752
cD
Coa
fl
O
?
CN
s
s
Sg
S
f
2
U.S. Patent
Oct. 7, 1997
sheet 4 of 52
5,675.752
FIG. 4a
EEA
1% Edit View Operations Tools
Help
Ctrl-F2
Save
Save AS...
Print...
Shift-F12
F2
Ctrl--Shift-F2
Print Setup.
Exit
Alt+F4
FIG. 4b.
EE
7
2 File 2Edit2 View Operations Tools
YL
Help
aaaaaaaaaaaaaaay
Replace
U.S. Patent
Oct. 7, 1997
Sheet 5 of 52
5,675,752
Status Bar
Screen Catalog
EE
/
2
File
Edit View
Operations2. Tools
Application Screen
Forward Path
Back Path
Ad Insertion
U.S. Patent
Oct. 7, 1997
Sheet 6 of 52
M
1. Ele E view operators (os)
5,675,752
He
Hel
Screen Editor
Screen Template Editor
Media Frame Editor
Design Check
Edit Mode
FIG. 4f
EEA
2 File
Edit View
Operations
Tools
U.S. Patent
5,675,752
U.S. Patent
5,675,752
IE|GTSpUuObEZ?SIG|UJTOZE)S
??
U.S. Patent
5,675,752
?º-1ZTE
EFG-ZTISLOU,
U.S. Patent
Oct. 7, 1997
Sheet 10 of 52
5.675,752
|<F.Ø->
?SU|BI(OT~_ÐZERAG)
U.S. Patent
Oct. 7, 1997
pue[80W
SuOI?BÐAWT
X_10Si8lºd
Sheet 11 of 52
5,675,752
U.S. Patent
5,675,752
BFTEÆS?ÐIGUOZLD
U.S. Patent
U30JOS
Oct. 7, 1997
Sheet 13 of 52
5,675.752
|JØDuÐSSIOJe?ud0 ASJ908pIA
028
U.S. Patent
D[EOW|ER$G#)1
Oct. 7, 1997
Sheet 14 of 52
5,675,752
U.S. Patent
Oct. 7, 1997
Sheet 16 of 52
5,675,752
Ø
U.S. Patent
5,675,752
U.S. Patent
[]
Oct. 7, 1997
Sheet 18 of 52
5.675,752
U.S. Patent
[]
5,675,752
U.S. Patent
Oct. 7, 1997
Sheet 20 of 52
|pue[29W9621 A0g
?!
[]
?!
!!
tae !
?!
?
L- No.?
5,675,752
U.S. Patent
5.675,752
ZTE,
U.S. Patent
Oct. 7, 1997
Sheet 22 of 52
HCJOEL EC]
ÔSWHNSON|W||(TEJOHEIONWS]
5,675,752
U.S. Patent
Oct. 7, 1997
Sheet 23 of 52
5,675,752
BGUEORZSLITDÐ
U.S. Patent
5,675,752
U.S. Patent
Oct. 7, 1997
o=u18ao?VT=
EXIW
2010 2025
SNO|WHE
Sheet 25 of 52
WT?ZéføØE
5,675,752
ZTILO-E>
2035 2030
U.S. Patent
Oct. 7, 1997
sheet 26 of 52
5,675,752
FIG. 17a
Tools
174411.7/11/wa/AAAA/47
Save
Save AS.
Shift-F12
F12
New Screen
Shift-F12
Open Screen Shift-F12
Print.
Crt-Shift-F12
Print Setup.
Exit
w
Alt+F4
BScreen Editoes,
CZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZM
1 tie (Eitz view to
- Delete
...
Delete All
be
al
Del
U.S. Patent
Oct. 7, 1997
Sheet 27 of 52
5,675,752
FIG. 17C
a
Screen Edito
is
Ele Eat (view2 roos
-
–
Template Mode
help
I-
Media Frame Catalog
Control Function Catalog
| Toolbar
Status Bar
FIG. 17d
w
EScreen
Editos. A
Z-4-4-4-4-4-4-4-4-4-4444-4-4-4-4-4-4-4-4-4-4-4-4-4-4-4-4-4
a Fe. Eve" ScreenTemplate
(c.2 Editor
aaaaaaaaaaaa
Eep a
comrad)
Media Frame Editor
Interactive Presentation Editor
Edit Mode
Test Mode
Escreen Editose. A
A File Edit View Iools
(Help2.
Help Index.
About...
U.S. Patent
Oct. 7, 1997
ØØ
Ø
Ø
ØEWTÓ
Sheet 28 of 52
5,675,752
U.S. Patent
Oct. 7, 1997
Sheet 30 of 52
5,675.752
N
|-|W
2600 2610
FETLØ->
U.S. Patent
Oct. 7, 1997
Sheet 31 of 52
5,675.752
FIG 21a
2
%controls2.
A
2631 (ZSliders/2-2632
2-2632 (indicators2
CIndicators
CButtons 9-2631
2633
20gos2
ZZZZZZZZZZZZZZZZZZZZZZZZZZZ
2615
26.17
WESTERNPACIFICAR #1
WESTERNPACIFICAR #2
BOB'S DISCOUNTAIR MAIN
BOB'S DISCOUNTAIR COMPRESSED
WORLD SYWAYS VERTICAL
WORLD SKYWAYS HORIZONTAL
2SELECf2.
2619
U.S. Patent
Oct. 7, 1997
Sheet 32 of 52
5,675.752
FIG. 21C
5%ggungs: Bg2,
VBackgungcolor
26ackgrieve MBrice
Zodesy,Motor Bessines,
see: 3
2637
THICK DROP BOX
THN DROPBOX
WERTICALDWIDER LINE
WIDE RAISED BOX
FIG. 21d
2%55.5%.
ZMedia
Bane/MegaFaroe Zassign, SalesIt
2640
2643
ORDER INSTRUCTOX2/ZZZZZZZ
PURCHASE REVIEW
REQUEST TO PASSENGER
STATEMENT 1
STATEMENT2
TRANSACTION VERIFICATION
WAIT MESSAGE/PROMPT
ress Purchase to
i rder
the item
2641
2642
U.S. Patent
Oct. 7, 1997
€383
Z 83
0383
BZZ
0 8Z
9483
Sheet 33 of 52
5,675.752
U.S. Patent
5,675.752
U.S. Patent
Oct. 7, 1997
Sheet 35 of 52
5,675.752
OZZ'9|-
[DTEZ×BI?]KXÆG>E|RT[S?H]
INCJWN
09831Z7
U.S. Patent
Oct. 7, 1997
Sheet 36 of 52
5,675.752
:
ANNA
O
e
O
ou
C)
n
S
Sa
s
S
n
5
g
d
O
to
O
O
H
(M)
C
9.
s
e
&B
h
c)
O
O
g
A
E
EEE
EEE
siggs
U.S. Patent
Oct. 7, 1997
Sheet 37 of 52
5,675,752
FIG. 23a
A
Edit View
Operations
Tools
Help
Ctrl-F2
Shift--F12
F2
Ctrl-Shift-F12
Print Setup...
Exit
Alt+F4
FIG. 23b
EE
Y Ele
View Operations Tools
Help
- 1.
--
U.S. Patent
Oct. 7, 1997
Sheet 38 of 52
5,675,752
FIG. 23C
E.
Operations Tools
1 Ele EditEdit (view2 Operations
Tools
A File
Help
Help
ZOOmln
ZOOm Out
Track Set...
FIG. 23C
Y
1 File Edit View (Operations) tools
Yaaaaaaaaaaa
Fade in
Fade Out
Dissolve
Vertical Wipe
Vertical Wipe Direction.
Horizontal Wipe
Horizontal Wipe Direction...
Diagonal Wipe
Diagonal Wipe Direction.
Set Breakpoint
Clear Breakpoint
Clear All Breakpoints
Set Maximum Time...
Help
Hazaa
M
U.S. Patent
Oct. 7, 1997
Sheet 39 of 52
5,675,752
FIG. 23e
o
2 File
Edit View Operations
Help
Zazaaaaaaaaaaaaanaan
Image Editor
Graphics Editor
Animation Editor
Video Editor
Sound Editor
TextEditor
Pick Editors...
Run Selected
Run A
Compile Media Frame
FIG. 23f
T
1
File
. . Edit
. . View
. . Operations
.
. Tools
. . . . . . .. .
.
A
1 - 1 - 7 -z-z-z-z-z-z-z-z-z-z-z-z-z-z-z-z-
Help Index.
AbOut...
U.S. Patent
Oct. 7, 1997
Sheet 40 of 52
5,675,752
FIG. 24a
FIG. 24b
- FIG.24C
FIG. 240
FIG. 24e
U.S. Patent
Oct. 7, 1997
Sheet 41 of 52
5,675,752
FIG.24f
s
FIG. 24g
FIG. 24h
FIG. 24
FIG. 24
U.S. Patent
Oct. 7, 1997
Sheet 42 of 52
s
5,675,752
FIG. 24k
FIG.24
FIG. 24m
FIG. 24n
5,675,752
U.S. Patent
L1[C-H ET?VSZI)
[]
/
U.S. Patent
Oct. 7, 1997
Sheet 45 of 52
5,675,752
R
SNN a
N
N
N
NN
e
N
N
NY
S
Sg
CD
cas
N
D
E
s
St
U.S. Patent
Oct. 7, 1997
MEDIA FRAMED
MEDIA FRAME SIZE
MEDIA TYPES COUNT
SOUND OFFSET
IMAGE OFFSET
GRAPHICS OFFSET
ANIMATION OFFSET
VIDEO OFFSET
SOUND ITEM COUNT
SOUND ITEM 1 TYPE
SOUND ITEM 1 OFFSET
SOUND ITEM 1 SIZE
SOUND ITEM 1 TIME
SOUND ITEM2 TYPE
SOUND ITEM2 OFFSET
SOUND ITEM2 SIZE
SOUND ITEM2 TIME
SOUND ITEM 1 DATA
SOUND ITEM2 DATA
Sheet 47 of 52
5,675,752
FIG. 29a
FIG.29b.
U.S. Patent
Oct. 7, 1997
Sheet 48 of 52
5,675,752
MAGE ITEM COUNT
WA
W
MA
W.
W.
IMAGE ITEM2 TYPE
WA
W
IMAGE TEM2 SIZE
IMAGE ITEM2 TIME
IMAGE ITEM 3TYPE
IMAGE ITEM 3 OFFSET
IMAGE ITEM 3 SIZE
IMAGE TEM3 TME
FIG. 29C
IMAGE ITEM 1 DATA
MAGE TEM2 DATA
IMAGE ITEM 3 DATA
RAPH
MCOUM
GRAPHICSTEM 1 TYPE
se
PAP
GRAPHICSTEM1 SIZE
GRAPHICS ITEM 1 TIME
G RAPHICSTEM1 DATA
FIG. 290
U.S. Patent
ANIMATION
ANIMATION
ANIMATION
ANEMATION
ANIMATION
ANIMATION
ANIMATION
ANIMATION
ANIMATION
Oct. 7, 1997
TEM COUNT
TEM 1 TYPE
TEM OFFSET
TEM SIZE
TEM TIME
TEM2 TYPE
TEM2 OFFSET
TEM2 SIZE
TEM2 TIME
Sheet 49 of 52
5,675,752
FIG. 29e
ANIMATION TEM 1 DATA
ANIMATION TEM2 DATA
VIDEO TEM COUNT
VIDEO TEM 1 TYPE
VIDEO ITEM 1 OFFSET
VIDEO TEM SIZE
VIDEO TEM 1 TIME
VIDEO TEM2 TYPE
VIDEO TEM2 OFFSET
VIDEO TEM2 SIZE
VIDEO TEM2 TIME
VIDEO. TEM DATA
VIDEO TEM2 DATA
FIG.29f
U.S. Patent
Oct. 7, 1997
Sheet 50 of 52
5,675,752
FIG. 30
SCreen
Template
Editor
Screen Editor
Intermediate
Format
Interactive Application Editor
Output Program Generator
End Use Specific
5E
y
U.S. Patent
Oct. 7, 1997
Sheet 52 of 52
5,675,752
5,675,752
1.
INTERACTIVE APPLICATIONS
GENERATOR FOR AN INTERACTIVE
PRESENTATION ENVIRONMENT
FIELD OF THE INVENTION
The present invention relates to the field of automated
software generation tools, and more particularly, the present
invention relates to an interactive applications generator for
use in an interactive presentation environment,
BACKGROUND OF THE INVENTION
In general, interactive applications software is developed
for use in an interactive applications environment. Typically,
interactive applications software involves the manipulation
of media and the presentation of a graphical user interface.
The interactive applications environment permits a user to
select various media elements via the graphical user inter
face. For example, in a cable television interactive system,
the customer may be presented with a user interface con
sisting of one or more menu screens. The menu screens
permit the user to select various media elements, such as
movies, music videos and television programming.
Preferably, the menu screens are customized for a particular
audience. In addition, it is desirable to update the menu
screens over a period of time. Upon entering a selection via
the user interface menu screen, the user is presented with the
appropriate media element. It is necessary to update the
10
15
20
25
The creation, modification, and maintenance of interac
30
35
Therefore, it is an object of the present invention to
provide an interactive applications software generation tool
It is another object of the present invention to utilize
interactive application object libraries to insert functionality
into the interactive applications generator.
These and other objects of the present invention are
realized in an arrangement that includes an interactive
applications generator that generates interactive application
programs for operation in an interactive application envi
ronment. In a preferred embodiment, the interactive appli
cation environment is implemented on a single server multi
client network computer system. The application programs
permit users, accessing the single server multi-client
network, to interact with multimedia presentations. The
single server multi-client network computer system includes
a graphical user interface (GUI) that defines the interactive
application environment. In general, the interactive applica
tion environment permits a user to access selective multi
media presentations. The interactive application environ
ment contains one or more display screens. In general, the
display screens are either menu screens or application
screens. The menu screens permit selection of additional
gram.
The GUI look and feel of the multi-client network com
45
55
puter system, including the presentation of display screens,
is generated by the interactive applications generator of the
present invention. The interactive applications generator
may be executed on any computer system equipped to
handle multi-media applications. In order to generate a
customized interactive application environment for a par
ticular application, the interactive applications generator
contains a screen template editor, a media frame editor, a
screen editor, and an interactive presentation editor. The
screen template editor is utilized to create generic screen
templates, that contain a plurality of generic screen template
elements. The media frame editor is utilized to generate
media frames by combining multimedia component ele
ments in a time synchronized manner. The screen editor is
utilized to generate the display screens including assigning
functionality. In order to create a unique instance for each
display screen for a particular interactive applications
environment, the screen editor references generic screen
templates and media frames. The interactive presentation
interactive applications generator to develop and maintain
interactive applications software for use in an interactive
presentation environment.
SUMMARY AND OBJECTS OF THE
INVENTION
update each individual display screen used by an interactive
application program.
It is another object of the present invention to provide an
automated method to interconnect display screens into inter
active application programs utilizing a drag and drop opera
menu screens or application screens, and the application
screens constitute a GUI for a particular application pro
to the new media elements. In addition, it is desirable to
update the customized user interface without the need to
update each individual display screen used by an interactive
application program. Consequently, it is desirable to sepa
rate the user interface portion of multimedia display screens
from the interactive media portion. As will be described, the
present invention provides methods and apparatus for an
updating the customized user interface without the need to
tion.
available media elements as new media elements become
available.
tive applications software is difficult and time consuming.
For example, each application for the interactive application
software involves development of a unique user interface so
that each customer receives a customized product having a
unique look and feel. Therefore, for each application, the
developer of the interactive applications software must
specialize each display screen in the graphical user interface
for each customer. Consequently, it is desirable to develop
tools to facilitate the development of customized display
screens for a user interface. Furthermore, the presentation of
menu screens to create the user interface is dependent upon
the particular application. Therefore, for each application,
the developer of the interactive applications software must
specialize the software to accommodate the appropriate
presentation of menu screens. Consequently, it is desirable
to develop tools to facilitate in the development of the
presentation of screens to create customized interactive
applications environments.
In addition to the development of interactive applications
software, the maintenance and updating of interactive appli
cations software is typically difficult. For example, in any
interactive applications environment, as new media ele
ments become available, the interactive applications soft
ware requires updating to accommodate selection and access
2
that minimizes the cost and difficulty in creating large
amounts of custom interactive software involving media.
It is a further object of the present invention to decouple
the user interface portion of multimedia display screens
from the interactive media portion.
It is another object of the present invention to permit
generation of customized look and feel user interfaces while
maintaining a generic base of interactive media.
It is another object of the present invention to permit
editor is utilized to create the hierarchical structures that
defines the presentation of the display screens for the
65
particular interactive presentation environment.
The screen template editor contains a screen template
editor graphical user interface (GUI) that displays a plurality
of tool dialog boxes on an output display to permit a user to
select the generic screen template elements. The screen
5,675,752
3
template editor GUI also contains a screen template build
window for designing the screen template. The tool dialog
boxes include a logos dialog box, a backgrounds and borders
dialog box, controls dialog box and a media frame window
dialog box. The controls dialog box permits selection of
buttons, sliders and indicators. The borders dialog box, the
logos dialog box, and the media frame windows dialog box
contain a list box, for providing options to define a graphical
look of the screen template, and a thumbnail sketch window
for displaying a corresponding highlighted selection in the
list box. In addition, a select button permits a user to select
the corresponding highlighted selection in the list box. The
borders dialog box permits a user to specify background
color, background texture, border color, border style, drop
boxes and lines, and the logos dialog box permit a user to
select from a number of available logos. The media frame
windows dialog box permits a user to select a media frame
window location within the generic screen template, such
that a dotted border comprising size and aspect ratio of said
media frame selected is displayed in the screen template
build window.
O
15
20
The media frame editor permits a user to generate media
frames comprising images, graphics, animations, video
clips, text, and sound clips. The media frame editor includes
a media frame editor graphical user interface (GUI) that
displays a timeline window consisting of media tracks, a
time scale and a vertical cursor. The media frame editor GUI
contains a media selection window, including a list box and
buttons, to permit a user to attach media frames to a media
track. The media selection window further includes a thumb
nail sketch window for displaying viewable media elements
selected via the list box. Arun window, contained within the
media frame editor GUI, permits a user to view selected
media frames. In order to run selected media flames, break
points are set so that the run window displays successive
media frames on a selected media track between adjacent
breakpoints. The media frame editor GUI also contains time
line transition special effect functions including a cut, fade
in, fade out, dissolve, horizontal wipe, vertical wipe, and
digital wipe. Furthermore, the media frame editor GUI
includespan, Zoom and complex pan and Zoom functions to
generate media frames that incorporates panning and Zoom
ing of a media element.
The screen editor contains a screen editor graphical user
interface (GUI) that displays a screen layout including a
graphical representation of the generic screen template ref
erenced. In addition to the borders and logos, the generic
screen template includes outlines of media frame windows
25
30
35
40
with references to the following drawings.
FIG. 1 illustrates a computer system for generating a
window-based graphic user interface in accordance with the
teachings of the present invention.
FIG. 2 illustrates a program hierarchy for the interactive
applications generator configured in accordance with the
present invention.
FIG. 3 illustrates a graphical user interface for an inter
active presentation editor configured in accordance with the
present invention.
FIG. 4a illustrates a file menu for the interactive presen
tation editor configured in accordance with the present
invention.
45
FIG. 4b illustrates an edit menu for the interactive pre
sentation editor program configured in accordance with the
present invention.
FIG. 4c illustrates a view menu for the interactive pre
sentation editor configured in accordance with the present
invention.
50
55
screen identifiers in a hierarchical structure. The interactive
presentation editor GUI includes a screen catalog to permit
a user to scroll through a library of display screens. The
screen catalog contains windows and controls to permit
BRIEF DESCRIPTION OF THE DRAWINGS
The objects, features, and advantages of the present
invention will be apparent from the following detailed
description of the preferred embodiment of the invention
and controls. The screen editor GUI contains a media frame
catalogue comprising a separate window that permits a user
to scroll through a catalogue of media frames. Utilizing the
media frame catalogue, the user assigns, via a drag and drop
operation, a media frame to a media frame area on the screen
layout. A function library catalog contains a separate win
dow that permits a user to scroll through a library of
predefined functions. The function library catalog is also a
drag and drop operation that permits the user to assign
functionality to a control on the screen layout.
The interactive presentation editor contains an interactive
presentation editor graphical user interface (GUI) that dis
plays screen identifier icons on an output display. The screen
identifiers correspond to the display screens, and the inter
active presentation editor GUI permits a user to place the
4
selection of a display screen for placement in the interactive
applications environment.
The interactive presentation editor GUI contains dialog
boxes corresponding to each display screen that allows a
user to enter specific display screen information such as a
text description, a screen identifier to link a display screen
to a media frame and a screen template, a list of each control
on the display screen, and a link to either an application
Screen or menu screen. In addition, grouping information, to
classify display screens in a group level, may be entered. In
order to create a hierarchical structure of display screens.
drag and drop menu screen and application screen opera
tions are provided to place a corresponding screen identifier,
representing a display screen selected, in any level of the
hierarchical structure. If a child display screen is dropped
within a parent display screen, then a forward link from the
parent menu screen to the child display screen is created.
Alternatively, a forward link command is provided to estab
lish a link from a control in a parent menu screen to a child
display screen. In addition, the interactive presentation edi
tor GUI contains a backpath command to establish link from
a child display screen to exit to a screen other than a parent
display screen.
65
FIG. 4d illustrates an operations menu for the interactive
media presentation editor configured in accordance with the
present invention.
FIG. 4e illustrates a tools menu for the interactive pre
sentation editor program configured in accordance with the
present invention.
FIG. 4fillustrates a help menu for the interactive presen
tation editor program configured in accordance with the
present invention.
FIG. 5 illustrates a screen identifier movement operation
via the interactive presentation editor.
FIG. 6 illustrates the interactive presentation editor
graphical user interface configured in accordance with the
present invention.
FIGS. 7a-billustrate an example of the expand command
configured in accordance with the interactive presentation
editor of the present invention.
5,675,752
5
6
FIG. 23d illustrates an operations menu for the media
frame editor configured in accordance with the present
FIGS. 8a-billustrate the reduce command for an example
screen hierarchy.
FIGS. 9a–b illustrate an example of the tree subset
invention.
FIG. 23e illustrates a tools menu for the media frame
function configured in accordance with the present inven
tion.
FIG. 10 illustrates an example of the functionality Of the
menu screen command.
FIGS. 11a-11b illustrate the operation of adding an appli
cation screen to a screen hierarchical structure in accordance
with the present invention.
10
forward path command for an example screen hierarchical
15
creation of the forward link between the ICS main menu
FIGS. 13a-b illustrate an example of a screen hierarchy
20
25
FIG. 17a illustrates a file menu for the screen editor
FIG. 17b iustrates an edit menu for the screen editor
30
tion.
FIG. 17c illustrates a view menu for the screen editor
35
configured in accordance with the present invention.
FIG. 17e illustrates a help menu for the screen editor
program configured in accordance with the present inven
FIG. 18 illustrates an example of a media frame catalog
window configured in accordance with the present inven
tion.
invention.
45
FIG. 20 illustrates a graphical user interface for the screen
template editor configured in accordance with the present
invention.
FIG. 21a illustrates a controls tool dialog box configured
in accordance with the present invention.
FIG. 21b illustrates a logos tool dialog box configured in
accordance with the present invention.
FIG. 21c illustrates a backgrounds and borders tool dialog
box configured in accordance with the present invention.
FIG. 21d illustrates a media frame windows tool dialog
box configured in accordance with the present invention.
FIGS. 22a-d illustrate a graphical user interface for the
media frame editor configured in accordance with the
present invention.
50
55
FIG. 23a illustrates a file menu for the media frame editor
FIG. 23b illustrates an edit menu for the media frame
FIG. 23c illustrates a view menu for the media frame
editor configured in accordance with the present invention.
to others skilled in the art.
An algorithm is here, and generally, conceived to be a self
quantities take the form of electrical or magnetic signals
capable of being stored, transferred, combined, compared,
and otherwise manipulated. It proves convenient at times,
principally for reasons of common usage, to refer to these
signals as bits, values, elements, symbols, characters,
images, terms, numbers, or the like. It should be borne in
mind, however, that all of these and similar terms are to be
associated with the appropriate physical quantities and are
merely convenient labels applied to these quantities.
In the present case, the operations are machine operations
performed in conjunction with a human operator. Useful
machines for performing the operations of the presentinven
tion include general purpose digital computers or other
similar devices. In all cases, there should be borne in mind
configured in accordance with the present invention.
editor configured in accordance with the present invention.
representations of operations of data bits within a computer
consistent sequence of steps leading to a desired result.
These steps are those requiring physical manipulations of
physical quantities. Usually, though not necessarily, these
tion.
FIG. 19 illustrates an example of a function library
catalog window configured in accordance with the present
The detailed descriptions which follow are presented
largely in terms of display images, algorithms, and symbolic
memory. These algorithmic descriptions and representations
are the means used by those skilled in the data processing
arts to most effectively convey the substance of their work
program configured in accordance with the present inven
tion.
FIG. 17d illustrates a tools menu for the screen editor
FIG. 28 illustrates a second complex pan and zoom effect
for the media frame editor of the present invention.
FIGS. 29a-fillustrate a target application media frame
format configured in accordance with the present invention.
FIG. 30 illustrates a generalized output format for the
interactive applications generator of the present invention.
FIGS. 31a-b illustrate one implementation, in Booch
notation, for the interactive applications generator.
NOTATION AND NOMENCLATURE
configured in accordance with the present invention.
program configured in accordance with the present inven
FIG. 27 illustrates a first complex pan and Zoom effect for
the media frame editor of the present invention.
screen and the information services menu screen.
structure for the operation of the back path command.
FIG. 14 illustrates an example of a screen hierarchical
structure during the design check process.
FIGS. 15a-billustrate an example screen hierarchy struc
ture for the operation of the regeneration command.
FIG. 16 illustrates a GUI for the screen editor configured
in accordance with the present invention.
FIG.25 illustrates a pan display effect for the media frame
editor for the present invention.
FIG. 26 illustrates a zoom display effect for the media
frame editor of the present invention.
FIG. 12a illustrates the creation of a forward link via the
Structure.
FIG. 12th illustrates the menu hierarchical structure after
editor configured in accordance with the present invention.
FIG.23fillustrates a help menu for the media frame editor
configured in accordance with the present invention.
FIGS. 24a-n illustrate the media frame track edit syntax
for a preferred embodiment of the present invention.
65
the distinction between the method operations of operating
a computer and the method of computation itself. The
present invention relates to method steps for operating a
computer and processing electrical or other physical signals
to generate other desired physical signals.
The present invention also relates to apparatus for per
forming these operations. This apparatus may be specially
constructed for the required purposes, or it may comprise a
general purpose computer selectively activated or reconfig
ured by a computer program stored in the computer. The
algorithms, methods and apparatus presented herein are not
inherently related to any particular computer. In particular,
various general purpose machines may be used with pro
grams in accordance with the teachings herein, or it may
5,675,752
7
prove more convenient to construct more specialized appa
ratus to perform the required method steps. The required
structure for a variety of these machines will appear from the
8
to be described more fully below. Moreover, in the presently
preferred embodiment, the present invention's window
based user interface is generated and displayed using soft
description given below. Machines which may perform the
functions of the present invention include those manufac
tured by Sony Trans Com, as well as other manufacturers of
ware stored in either memories 26, 32 or CD ROM 34, and
executed by the CPU 24.
In the presently preferred embodiment, the cursor control
52 utilizes well known apparatus and methods for signaling
CPU 24 of position changes of cursor 58 by movement of
computer systems.
DETAILED DESCRIPTION
The present invention discloses apparatus and methods
for an interactive applications generator. In the following
description, numerous specific details are set forth such as
computer system configurations, window elements, icons,
desktop sizes, metaphors, window configurations and
arrangements, etc. in order to provide a thorough under
standing of the present invention. However, it will be
apparent to one skilled in the art that the present invention
may be practiced without these specific details. In other
instances, well known circuits, structures and the like are not
described in detailso as not to obscure the present invention
unnecessarily.
Referring to FIG. 1, the hardware configuration of the
present invention is conceptually illustrated. FIG. 1 illus
trates a computer system for generating a window-based
graphic user interface in accordance with the teachings of
10
15
20
the cursor control over a surface. However, it will be
appreciated by one skilled in the art that a variety of well
known cursor control devices may be utilized by the present
invention, including other control devices such as mechani
cal mice, trackballs, joy sticks, graphic tablets, other key
board inputs. The cursor control 52 in FIG. 1 is intended to
encompass all such equivalent devices.
The methods and apparatus of the present invention are
utilized to generate application programs for operation in an
interactive applications environment. An interactive appli
cations environment may be implemented on any single
server multi-client network computer system. In a preferred
embodiment, the application programs permit users, access
ing the single server multi-client network, to interact with
multimedia presentations. The interactive applications envi
ronment
includes a graphical user interface (GUI) that
25
permits a user to interact with multimedia presentations. The
the present invention. As illustrated, the computer system graphical user interface contains one or more display
includes a computer 20 which comprises four major com screens. In general, the display screens are either menu
ponents. The first of these is an input/output (I/O) circuit 22, screens or application screens. The menu screens permit
which is used to communicate information in appropriately 30 selection of additional menu screens or application screens,
structured form to and from other portions of the computer and the application screens constitute a GUI for a particular
20. In addition, computer 20 includes a central processing application program.
unit (CPU) 24 coupled to the I/O circuit 22 and to a memory
The present invention permits generation of customized
26. These elements are those typically found in most com interactive applications environments through use of an
puters and, in fact, computer 20 is intended to be represen 35 interactive applications generator. The interactive applica
tative of a broad category of data processing devices.
tions generator provides a means for a user to create a
Also shown in FIG. 1 is a keyboard 30 for inputting data customized interactive applications environment for a par
and commands into computer 20 through the I/O circuit 22, ticular application. For example, an interactive applications
as is well known. It will be appreciated that additional environment may be generated for an airline passenger
devices may be coupled to the computer 20 for storing data, 40 entertainment system. Specifically, the interactive applica
such as magnetic tape drives, and the like. A device control tions generator permits generation of customized logos,
36 is coupled to both the memory 26 and the I/O circuit 22, background colors and textures, graphical design style of
to permit the computer 20 to communicate with multi-media controls including buttons and sliders, and a generalized
system resources. The device control 36 controls operation look and feel of a graphical user interface for a particular
of the multi-media resources to interface the multi-media 45 application.
resources to the computer 20. For example, in an interactive
The graphical user interface elements are derived from a
applications generator environment utilizing video, the com generic screen template and are created through use of a
puter 20 may be coupled through the device control 36 to a screen editor. The generic elements of a screen template
video tape recorder 40, compact disc read only memory (CD include a background, controls, corporate logos, and display
ROM) 42, an audio tape recorder 44, and a disc player 46. windows for displaying text and graphical information. Each
The device control 36 may comprise an independent control display screen contains a corresponding screen template. In
and interface device or a software process for execution by addition to screen templates, a display screen may also
the CPU 24.
contain a media frame. The media frame is displayed over a
A display monitor 50 is coupled to the computer 20 portion of the display screen or over the entire display
through the I/O circuit 22. Any well known variety of 55 screen. The screen templates contain all of the graphical user
cathode ray tube (CRT), liquid crystal or other displays may interface elements required to customize a display screen for
be utilized as display monitor 50. A cursor control device 52 a particular application. The controls located on the screen
includes switches 54 and 56 for signaling the CPU 24 in templates provide functionality for the interactive applica
accordance with the teachings of the present invention. tions environment.
Cursor control device 52 (commonly referred to as a
The media frames are generated by the interactive appli
"mouse”) permits a user to select various command modes, cations generator via a media frame editor. The media
modify graphic data, and input other data utilizing switches utilized in the media frames may consist of any multimedia
56 and 54. More particularly, the cursor control device 52 subject matter. For example, in an airline passenger enter
permits a user to selectively position a cursor 58 at any tainment system, the multimedia subject matter may include
desired location on a display screen 60 of the display 50. As 65 duty free shopping product advertisements, photographs of
illustrated in FIG. 1, the cursor 58 is disposed within a a destination city, and destination city maps. In addition to
window 65 in the present invention's graphic user interface, the multimedia subject matter, each media frame may com
5,675,752
prise one or more types of multimedia. For example, a media
frame may comprise a graphical image, a video clip, an
animation sequence, a sound clip, graphics, and blocks of
text with display attributes. As will be explained more fully
below, all multimedia subject matter associated with a
particular media frame is loosely synchronized to a common
time line so that each media frame contains exactly one time
line.
Each display screen has a unique existence within a
particular interactive applications environment. However,
both screen templates and media frames may be shared by
multiple instances of display screens. For example, in the
airline passenger entertainment application, a catalog shop
ping application program is constructed to randomly change
sequences of screen templates to create multiple unique
instances of display screens. For media flames, an image of
a product may be used with multiple instances of display
screens, wherein each display screen contains separate text
targeted at different audiences.
The airline passenger entertainment system provides one
example of an interactive applications environment gener
ated by the interactive applications generator of the present
invention. Although the airline interactive passenger enter
tainment system is provided as an example of an interactive
applications environment, many other possible applications
exist that share component elements similar to the airline
interactive passenger entertainment system. Such interactive
applications environments include business training
systems, educational systems for schools, library informa
tion systems, interactive cable or satellite home delivery of
multimedia and television, on line equipment repair infor
mation systems, health information systems, hospital patient
entertainment systems, cruise ship passenger entertainment
systems, bus and commuter train interactive multimedia
delivery systems. In addition, applications exist that require
screen templates and or media frames as defined in the
10
a project have a common name, but each of the files has a
15
20
25
30
35
In a preferred embodiment, the interactive applications
generator of the present invention is a software tool for
creating and modifying application programs for use in an
integrated interactive applications environment. The inter
active applications generator contains an interactive presen
tation editor, a display screen editor, a screen template
editor, and a media frame editor. In addition, the interactive
45
tion editor allows the user to create hierarchical structures of
screens. The screen editor permits the user to reference a
screen template and or media frames to create a unique
instance of a display screen. The screen editor also permits
the user to attach controls to a particular screen display. The
screen template editor permits a user to create screen tem
plate definitions. The media frame editor permits a user to
combine multimedia component elements in a time synchro
nized fashion to produce a media frame. The four editors
within the interactive applications generator also permit
modification of existing application programs and screens.
In a preferred embodiment, a design rules checker and an
application tester are utilized. The design rules checker
providers a user with a means for logically testing an
interactive applications environment to verify that the inter
nal links and attributes are consistent throughout the inter
active applications environment. For example, the design
rules checker identifies one way cul-de-sac paths or com
pletely unlinked screens. The applications tester provides a
unique file extension. A user of the interactive applications
generator can open only one instance of the interactive
present invention.
applications generator includes components for testing and
checking the operation of the interactive applications envi
ronment prior to generation of a target output for the
application programs. In general, the interactive presenta
10
demonstration for the user so that the user may check the
aesthetic functionality of an interactive applications envi
ronment. Specifically, the applications tester allows an
operator to identify a confusing screen sequence, timing
problems or inappropriate labeling of controls.
The interactive applications generator provides a means
for organizing work created in "projects”, wherein each
project identifies a separate interactive applications environ
ment. A new project is opened to begin creating a new
interactive applications environment. Opening a new project
creates a set of files to store the component elements of a
project. Each editor, within the interactive applications
generator, contains a file for the project. All of the files for
50
presentation editor. However, multiple instances of the
screen editor, screen template editor, and media frame editor
may be opened simultaneously. This configuration permits
flexibility such that a user may work on the component
elements of a single project but the user is precluded from
opening more than one project at a time. The interactive
applications generator permits a user to begin at any cat
egory block except for the design rules checker and the
applications tester.
FIG. 2 illustrates, in Booch notation, a program hierarchy
for the interactive applications generator configured in
accordance with the presentinvention. The interactive appli
cations generator contains four primary component pro
grams: an interactive presentation editor 200; a screen
template editor 210; a screen editor 220; and a media frame
editor 230. The interactive presentation editor 200 is shown
in FIG. 2 as a top layer for the interactive applications
generator. In order to create an interactive applications
environment for a particular application, a user may begin a
project utilizing a top to down design methodology by
beginning with the interactive presentation editor 200. In
general, the interactive presentation editor 200 permits a
user to build a skeleton to define a screen hierarchy.
Alternatively, a user may begin a project utilizing a bottom
to top design methodology by first creating the screen
templates and media frames. The interactive applications
generator provides menu selections for each of the four
primary component programs to allow: invoking an
instance; returning to an existing instance; or returning to
any of the other primary component programs. A windows
group box with icons for each of the four primary compo
nent programs provides flexible entry access to the interac
tive applications generator.
The interactive presentation editor of the present inven
tion provides the capability to link together a set of menus
and application screens to create a particular interactive
applications environment. Specifically, the interactive pre
sentation editor permits the operator to organize and display
55
screens in a screen hierarchy. In the interactive presentation
editor, the screens are represented as symbols for manipu
lation by the user. The screen hierarchy establishes paths for
the display screens to define an interactive applications
environment. The screen hierarchy is organized in hierar
chical levels, such that a user selects a screen at one level of
65
the hierarchy to permit entry into subsequently more
detailed levels of the hierarchy. For purposes of explanation,
child display screens are defined as those display screens
having a parent display screen located in the next higher
hierarchical level. In order to establish the screen hierarchy
for a particular interactive applications environment, each
child display screen is linked to a control on the respective
5.675,752
11
parent display screen. In addition, paths, known as back
paths, for exiting the display screen are also created. These
back paths typically return the user to a higher level in the
hierarchy.
FIG. 3 illustrates a graphical user interface (GUI) for an
interactive presentation editor configured in accordance with
the present invention. In a preferred embodiment of the
present invention, a GUI is generated on the computer
system via operation in the MicrosoftTM Windows operating
system environment. The GUI for the interactive presenta
tion editor 200 contains a title bar 300, menu bar 310, system
12
an example of a four level screen hierarchy for an airline
passenger entertainment interactive environment. The hier
archical path begins with a "welcome” menu display screen.
The "welcome” menu display screen permits a user to select
the "ICS main” menu or the "customization' menu. From
the ICS main menu, the entertainment, meal and beverage
10.
menu button 312, and minimize and maximize buttons 314
and 315. respectively. In addition, the GUIfor the interactive
presentation editor contains a tool bar 318 containing a
number of icons. The icons are used to represent commonly
used functions such as the rules checker and the applications
tester. The display of the tool bar 318 is toggled either to an
"on" or “off” state. A status bar 320 is used to display
informational messages to aid the operator. For the GUI
shown in FIG. 3, a description for the icon under the cursor
15
cations in a fifth hierarchical level. Also shown in the fifth
control device, the current Zoom ratio, and Zone restrictions
are shown. In addition, scroll bars 325 permit panning of the
graphical display in both horizontal and vertical directions.
Adepth ruler 330 displays the hierarchical levelin the screen
hierarchy for the corresponding display screens.
The interactive presentation editor 200 utilizes display
screen identifiers to graphically represent display screens for
arrangement by a user to create a particular interactive
applications environment. Generally, display screens consist
of menu screens and application screens. A menu display
screen permits selection of another menu display screen or
an application display screen. An application screen is a GUI
for a particular application program. A rectangular display
screen identifier represents a menu screen, and an oval
display screen identifier represents an application display
screen. In order to select a display screen from the interac
tive presentation editor 200, the user selects the respective
display screen identifier through use of a cursor control
device. When a particular display screen is selected, a dialog
box for the corresponding display screen is invoked. For a
menu display screen, the dialog box permits a user to enter
specific display screeninformation. For example, a user may
enter: the text description of the screen; a screen identifier
linking the screen to a media frame and a screen template;
a list of each control; and a link to either an application or
sub-menu screen. In addition, other attributes defining clas
sifications for use of the display screen may be entered.
Furthermore, information may be automatically entered into
the database without use of the dialog box if a display screen
is selected from a screen catalog. The operation and use of
a screen catalog is described more fully below.
If the menu or application display screen is linked to
additional menu and/or display screens, then the menu or
applications display screen identifier contains "a three
dimensional shadow”. For example, the meal and beverage
menu display screen identifier illustrated in FIG. 3 contains
additional menu and application display screens. When
viewing the screen hierarchy via the interactive presentation
editor, one interactive presentation path is highlighted. For
the example screen hierarchy illustrated in FIG. 3, the
display screens: “welcome”, “ICS main”, “entertainment",
and "games", that comprise the active interactive presenta
25
level are arrows, labeled 327 and 329, that permit the user
to scroll through the screen identifiers in either a right or left
direction, respectively. For groups of selected menu and or
application screens, double-clicking on any of the selected
display screens permits a user to enter information pertain
ing to a group level. For example, a user may set restrictions
or availability of certain screens based on a group level
classification. For the airline passenger entertainment
system, restricted access may be categorized by area Zone,
flight phase, age restriction, and special passenger informa
tion.
30
The interactive presentation editor of the present inven
tion determines the association of child screens to menu
35
40
parent display screens automatically. The determination is
based on the left to right ordering of the application screens
in the hierarchy, and the ordering set forth for each control
when the corresponding screen template is created. If too
many controls or too many child screens are associated with
a parent screen, the operator of the interactive applications
generator is prompted to correct the error during the design
check phase.
FIG. 4a illustrates a file menu for the interactive presen
tation editor configured in accordance with the present
invention. The file menu selection on the interactive pre
sentation editor GUI permits a user to open a new file. Upon
invocation of the “new” function, a windows common
45
50
55
dialog box is invoked to open a new file. An “open" function
permits a user to open a new or existing file within the screen
hierarchy. The interactive presentation editor files are given
the file extension “IPE”. When the file open function is
selected, only files ending with the “IPE" extension are
displayed for selection by the user. A "close" function in the
file menu closes the active file currently opened. In order to
avoid discarding updated information during an editing
session, a user is prompted to save changes if edits were
made to the file subsequent to the last save operation. A
“save” function permits a user to save the currently opened
file to disk. When selected, a windows common dialog box
is invoked to save the active "...IPE' file.
A "save as' function in the file menu permits a user to
save the active IPE file under a new name. A file extension
of IPE is automatically added to the name selected. A print
function within the file menu permits a user to print the
active screen hierarchy or a subset of the current screen
hierarchy. Options for the print function include printing the
current view on display, the current tree hierarchy being
tion path, are highlighted.
If a user selects an application screen, a dialog box is
invoked, permitting entry of application specific informa
tion. The interactive presentation editor 200 allows a user to
link menu screens to application screens. FIG. 3 illustrates
and passenger menu screens are selectable. From the enter
tainment menu display screen, a number of application
display screens are selectable. For the example shown in
FIG. 3, an audio, video, games, pay-per-view video and
pay-per-view games application screens are shown associ
ated with the entertainment menu display screen. If the user,
via a cursor control device, double-clicks on the games
application display screen, a list box, showing one or more
screen identifiers describing a set of screens for the selected
application, are displayed.
FIG. 3 illustrates screen identifiers for the games appli
65
edited, or a set of selected display screens. A "print setup"
function within the file menu permits a user to set up the
printer environment. Finally, the "exit" function within the
5,675,752
13
file menu permits exiting the active open file within the
interactive presentation editor. If the active IPE file has not
been saved upon invocation of the exit function, the user is
prompted to save the file before exiting.
FIG. 4b illustrates an edit menu for the interactive pre
sentation editor program configured in accordance with the
present invention. The edit menu contains an "undo' func
tion command that allows a user to repeatedly back out of
the last "n" operations up to a point. The number of undo
operations is a user defined option such that the user may
pre-configure the number in a configuration program. The
interactive presentation editor program edit menu also
14
cation screens contain controls that are not defined to the
10
includes a standard windows 'cut' function. The cut func
tion allows the user to cut or delete logical items in the
program. The deleted items are placed in the windows clip
board or buffer. The cut function permits a user to select
screen menus, including the screen menu associated
hierarchies, for removal or relocation. The edit menu also
displays a standard windows copy function which allows a
user to copy logical items in the program. Upon invocation
of the copy function, the logical items are placed in the
windows clipboard. For the interactive presentation editor
program, the copy operation copies selected screen menus,
including the screen menu associated hierarchies, to other
hierarchies, In addition, the copy operation may be utilized
as an editing shortcut when a portion of the screen hierarchy
is similar.
The edit menu also includes a standard windows "paste"
function, allowing a user to paste or insert logical items
currently stored in the windows clip board. A "delete"
function is also included in the edit menu. The delete
function is analogous to the cut function, except that the
information is not copied to the clip board. Items in the
interactive presentation editor program may also be deleted
by selecting the object and pressing a delete key on the host
computer keyboard. The "find" function, selectable within
the edit menu, permits a user to search for a particular
display screen. A user may search on the screen text descrip
tion or a screen identifier. If the display screen sought is
found, the screenis selected and displayed on the monitorso
that the selected screen appears within the current view
window. A “replace" command, selectable from the edit
menu, allows execution of a search and replace function in
the interactive presentation editor program. For example, the
replace function permits scanning of all display screens for
a given media frame for replacement with another. Confir
mation of each replacement function selected and global
replacement is an option.
FIG. 5 illustrates a screen identifier movement operation
via the interactive presentation editor. For the example
shown in FIG. 5, the user desires to move application
screens Pay-per-View Video and Pay-per-View Games to a
next higher hierarchical level. First, the application screens,
pay-per-view video and pay-per-view games, are selected as
indicated by the dotted rectangular box labeled 500 on FIG.
5. The selected application display screens are then cut via
15
20
25
"zoom” function, selectable from the view menu, that allows
a user to Zoom in or out of the current display window. The
user controls the Zoom function through use of a zoom scale
factor. The Zoom function enables the user to focus upon a
selected portion of the current output display. The interactive
presentation editor scales the selected portion to fit the
display window. The "expand” command, also selectable
from the view menu, allows a user to expand a menu screen
group or an application screen group. A selected menu
screen group is graphically expanded into a sub-menu
structure. The level of expansion into the sub-menu structure
depends upon an option initially set by the user. If expanded
to show all levels, then each hierarchical level stops at a final
menu screen or an application group. Alternatively, a
selected application group expands into a separate screen.
The selected application group is displayed showing the first
“n” application screens, with forward and backward buttons.
30 The direction buttons allow navigation through the set of
screens comprising the application group.
FIGS. 7a-billustrate an example of the expand command
configured in accordance with the interactive presentation
editor of the present invention. FIG. 7a illustrates an
35 example of a screen hierarchy for an interactive presentation
environment. To expand the meal and beverage menu dis
play screen, the user, through use of the cursor control
device, selects the meal and beverage menu display screen
identifier on the output display. FIG.7b illustrates the output
display after execution of the expand command on the meal
and beverage menu display screen identifier.
A reduce command, selectable from the view menu,
allows a user to reduce a selected menu group or application
group in the screen hierarchy. The reduce command allows
45 a user to concentrate on a particular area of interest without
viewing overall complexities in the entire screen hierarchi
cal structure. FIGS. 8a-billustrate the reduce command for
an example screen hierarchy. For the example illustrated in
FIG. 8a, a user selects a meal and beverage menu display
50
screen. Upon selection of the meal and beverage display
screen, a user selects the reduce command from the view
menu. After the user selects the reduce command, the
55
the cut function, and are subsequently stored in the windows
clipboard. A user then selects a new parent screen such as the
pay-per-view entertainment menu display screen. Upon
selection of the menu display screen, the interactive presen
tation editor joins the screens graphically as shown in FIG.
6.
The interactive presentation editor GUI illustrated in FIG.
6 connects, via a dotted line, the pay-per-view entertainment
menu screen to the pay-per-view video and pay-per-view
games application display screens. The dotted line indicates
that the pay-per-view video and pay-per-view games appli
parent screen. The placement of the pay-per-view video and
pay-per-view games application screens automatically
expands the corresponding hierarchical level to show the
new placement. Because only one branch of a parent menu
is expandable at any one particular time, the audio, video
and games application screens were reduced into the enter
tainment menu screen, and the entertainment menu display
identifier was changed accordingly.
FIG. 4c illustrates a view menu for the interactive pre
sentation editor configured in accordance with the present
invention. The interactive presentation editor contains a
65
physical portion of the hierarchical screen display is reduced
to the screen displays as shown in FIG. 8b.
A "backpaths' function is selectable from the view menu
as shown in FIG. 4c. In a preferred embodiment of the
present invention, back paths are defined as routes, within
the menu screen hierarchy, that a user may utilize to directly
exit a particular screen menu selected. Because viewing
back paths may confuse the overall screen hierarchical
structure, the output display does not show the backpaths
when operating in the default mode. In addition, the back
paths are always shown in an attention color, such as red.
The back paths function toggles the display of back links,
associated with a menu display screen hierarchy, on and off.
A levels function, within the view menu, toggles the
display of the depth ruler display "on" and “off”. A check
5,675,752
15
mark is displayed beside the levels function in the view
menu when the depth ruler is displayed. A tool bar
command, selectable within the view menu toggles the
display of a tool bar display “on” and “off”. A check mark
appears beside the toolbar function on the view menu when
the tool bar is displayed. A status bar command, selectable
from the view menu, toggles the display of the status bar
“on” and "off". A check mark is placed beside the "Status
Bar” function when the status bar is displayed. A screen
catalog function, selectable from the view menu, activates
and deactivates a screen catalog window. The screen catalog
window permits a user to scroll through a screen set for a
particular application. A user may select a screen within the
screen set display for insertion into the current screen
hierarchical structure. The selection of a screen via the
screen catalog window permits the interactive presentation
16
screen command. For the example illustrated in FIG. 10, an
information services menu screen is added. Note, because
the new menu screen was not dropped onto an existing menu
screen, the new menu screen is unattached. Because of this,
a forward link is established at a later time.
An application screen command, selectable within the
operations menu, is shown in FIG. 4d. The application
screen command allows a user to add a new application
screen to the screen hierarchical structure. As discussed
10
15
editor to maximize the amount of information set automati
cally. A screen catalog configured in accordance with the
present invention is described more fully below.
A tree subset function is also selectable from the view
menu shown in FIG. 4c. The tree subset function permits
20
restriction to menu screens based on classification informa
tion. The tree subset command permits a user to view a given
sub-diagram of the overall menu screen hierarchical struc
ture. The sub-diagram is generated by selecting one or more
restriction options. The tree subset function allows a user to
easily edit and check the menu screen hierarchy for each
classification or category. FIGS. 9a-billustrate an example
of the tree subset function configured in accordance with the
present invention. FIG. 9a illustrates an example of an adult
category for a layout of a menu screen hierarchical structure.
Note that the selected application screen (e.g. videos) shows
25
30
a number of videos suitable for an adult. Also note that the
status line 320 indicates that the menu screen structure
corresponds to a first class adult passenger. Upon selection
of a new tree type, via the tree subset command, the
interactive presentation editor revises the menu hierarchy
accordingly. FIG.9billustrates an example of a menu screen
hierarchical structure for a first class child passenger. The
entertainment options requiring money transactions and
specific adult oriented movies were removed from the output
display screen, thereby making these selections non
the same manner as a menu screen.
35
45
number of commands to allow the user to add a new menu
screen, establish a link, and insert a display screen. As
discussed above, a menu screen contains controls that permit
a user to navigate through the particular interactive appli
cation environment. For example, menu screens are used to
group related items together in the screen hierarchical struc
ture. A menu screen command, selectable from the opera
tions menu, is a drag and drop type command. Upon
invoking the menu screen function and selecting a menu
screen, the selected menu screen may be placed at any level
50
with the present invention. For the example shown in FIG.
11a, a chess application screen is selected, via the applica
tion screen command, and the corresponding screen identi
fier is attached to the cursor so that the user may place the
application screen in an application group. For the example,
the chess application screen is placed within the games
application group. FIG. 11b shows the output display after
the chess application screen is added to the games applica
tion group. Note that the chess application screen appears at
the far right of the application group, and the interactive
presentation editor program automatically scrolls all other
application screens to the left.
A forward path command, selectable from the operations
menu, is shown in FIG. 4d. The forward path command
establishes a link from a control in a menu screen at a “n”
55
within the screen hierarchical structure. After the selected
menu screen is placed, the user, via a dialog box, may enter
information pertaining to the selected menu screen.
Moreover, information may be added by the user at a later
time by double-clicking on the screen identifier for the
particular menu screen with the cursor control device.
If a menu screenis dropped within the parent menu screen
(i.e. the placement of the new menu screen identifier over
laps a parent menu screen display), then a forward link is
automatically established. If desired by the user, parent
menu screen information may be entered at this time. FIG.
10 illustrates an example of the functionality of the menu
FIGS. 11a–11b illustrate the operation of adding an appli
cation screen to a screenhierarchical structure in accordance
accessible to a child.
An operations menu for the interactive media presentation
editor is shown in FIG. 4d. The operations menu provides a
above, an application screen is located at the final destina
tion of a given menu path in an interactive applications
environment. Application screens are commonly used in
conjunction with other application screens. An application
screen contains forward and backward control buttons,
wherein the function of these buttons is predefined to enable
the user to move through the group of application screens.
In a preferred embodiment, application screens, contained
in a particular application screen group, exhibit a left to right
ordering. The left to right ordering defines the movement
through the application screens in a particular application
screen group. For example, in the airline passenger enter
tainment application, a duty free shopping application group
may contain many application screens for sale of different
items. The application screen command is a drag and drop
type command, whereby the application screen may be
placed at any level within the screen hierarchical structure.
After the application screen is placed within the screen
hierarchical structure, the user may enter information per
taining to the application screen either now, or at a later time.
A parent screen is automatically assigned to the selected
application screen based on the application group. If the
selected application screen was a first screen within a
particular application group, a connection is established in
level to a menu or application screen at a "n+1" level.
Although the forward path command connects application
and menu screens to controls, the standard drag and drop
procedure is the preferred procedure for making a forward
path link. FIG. 12a illustrates the creation of a forward link
via the forward path command for an example screen
hierarchical structure. For the example screen hierarchical
structure, a forward link is created between an information
services menu screen and an ICS main menu screen. The
65
forward link is established by selecting the two associated
menu screens with the cursor control device. In response to
selecting the forward path command, a mock screen tem
plate of the parent menu is graphically created allowing the
user to pick the control from the screen template. The
selection of the control on the parent menu screen estab
lishes the link to access the child menu screen. FIG. 12b
5,675,752
17
illustrates the menu hierarchical structure after creation of
the forward link between the ICS main menu screen and the
information services menu screen.
For a typical application, when a child screen is exited, the
program returns control to a parent screen. The return of
control to a parent screen from a child screen is the default
mode of operation in setting up an interactive applications
environment with the interactive applications generator,
However, in some cases, it is desirable to jump back several
levels to exit a particular child screen. Therefore, the present
invention provides a backpath command selectable from the
10
operations menu that allows a child screen to exit to a screen
other than the parent screen. The back path command
establishes a back link as an override to the default mode of
operation. To establish a back link, source and destination
screens are selected by the cursor control device.
15
FIG. 13a illustrates an example of a screen hierarchy
structure for the operation of the back path command. For
the example shown in FIG. 13a, an entertainment menu
screen and a welcome menu screen are the source and
destination screens, respectively, for the back path com
mand. The operation of the back path command to create a
back path is analogous to the operation of the forward path
command explained above. Once a back link has been
established, and if a forward link exists, the forward link is
changed from a solid line to a dashed line as shown in FIG.
13b. The dashed line represents the one way nature of the
forward link. In addition, the back link path is only shown
when a viewfback path option is selected.
An advertisement (ad) insertion command, selectable
from the operations menu, permits the user to insert an
20
25
30
advertisement before a particular screen is displayed. The
advertisement begins each time the associated screen is
selected. Upon completion of the advertisement, the asso
ciated screen is displayed. For example, for the airline
passenger entertainment system, an "In-flight games are
brought to you by XYZ Enterprises,” advertisement may be
displayed before entering into a games menu screen. When
a path to a screen contains an associated advertisement, the
screen icon is displayed with a 'S' icon.
FIG. 4e illustrates a tools menu for the interactive pre
sentation editor program configured in accordance with the
present invention. A screen editor command is shown select
35.
template editor command allows the user to create/edit the
actual screen templates while working within the screen
hierarchy. A menu frame editor command is also selectable
from the tools menu. Similarly, the media frame editor
allows a user to access the media frame editor program while
operating within the interactive presentation editor program.
Auser may wish to createfedit the actual media frames while
working within the screen hierarchy.
A design check command, selectable from the tools menu,
invokes the design check program. The design check pro
gram is used to perform a variety of information and error
checks within the current screen hierarchy. The design check
ment menu screen does not have controls attached, and
therefore is highlighted.
FIG. 4e illustrates a regenerate command selectable from
the tools menu. The regenerate command regenerates the
screen hierarchy after edits occur. For example, after editing
and moving the screen identifier icons, the resulting screen
hierarchy may be displayed having links that are criss
crossed. Upon invocation of the regenerate command, a new
graphical representation in a top-to-bottom, left-to-right
ordering is displayed. FIG. 15a illustrates an example screen
hierarchy after editing, but prior to execution of the regen
eration command. Note that the Information Services menu
chical structure.
An edit mode command is shown selectable from the
tools menu in FIG. 4e. The edit mode command sets the
mode of operation for the interactive presentation editor
program. The edit mode is the normal mode of operation for
the program, and permits the user to edit the current design.
The edit mode is used to return from the testmode. When the
interactive presentation editor operates in the edit mode, a
check mark resides beside the edit mode command on the
50
tools menu. Also shown on the tools menu in FIG. 4e is a test
mode command. The testmode command switches the mode
of operation from the edit mode to the test or demonstration
selectable from the tools menu is a screen template editor
command. The screen template editor command provides
access to the screen template editor program. The screen
does not have a forward and backward link from another
screen within the screen hierarchy. The design rules used by
this design check program can be flexibly configured via a
dialog box.
FIG. 14 illustrates an example of a screen hierarchical
structure during the design check process. When an error is
detected by the design check program, a display box, such
as display box 1700, is displayed. The display box 1700
identifies any errors detected. In addition, the display box
permits a user to fix, continue or cancel the design check
program. For the design check program, a screen identifier
icon is highlighted to identify the screen where the error was
detected. For the example shown in FIG. 14, the entertain
screen and the display menu screen are displayed within
menu screens having a different parent menu screen. FIG.
15b shows display of an example screen hierarchy after
execution of the regeneration command. The screen hierar
chy displayed in FIG. 15b more clearly shows the hierar
able from the tools menu. The screen editor command 45
permits a user to access the screen editor program. For
example, a user may wish to create/edit an actual menu
screen or application screen when working within the screen
hierarchy structure. Although the screen editor is typically
run separately, providing access to the user during operation
of the interactive presentation editor program is desirable to
provide greaterflexibility in the design process. Also shown
18
program identifies potential problems including identifying
missing information and circular menu screen paths. For
example, the design check program identifies menu screens
that are not attached to a specific control in a corresponding
parent screen. The design check program also identifies
controls in a menu screen that do not have a corresponding
application or menu screen attached. Furthermore, the
design check program identifies screens that have not been
linked to a parent screen. Finally, the design check program
identifies screens situated in endless loops, such that a screen
modes. A checkmarkresides beside the testmode command
on the tools menu when the interactive presentation program
55
is in the test mode.
FIG. 4f illustrates a help menu for the interactive presen
tation editor program configured in accordance with the
present invention. The help menu displays a “Help Index.
. . " and "About” commands. The Help Index command is
a standard windows help index type function. Generally, the
65
help index command permits a user to select from a category
index to obtain help information for operation of the inter
active presentation editor program. The about command is
also a standard windows type about box utilized for win
dows programs. The aboutbox, when selected, displays title,
copyright, and version information pertaining to the inter
active presentation editor.
5.675,752
19
The screen editor program of the present invention pro
vides capabilities to create screens utilized in the interactive
applications environment. A screen contains two parts: a
reference to a generic screen template, and a set of infor
mation specific to the screen (actual controls and media
frames) that customizes the screen template for a unique
interactive applications environment. To generate a screen
with the screen editor program, a screen is first associated
with a screen template. A window on the output display
displays a graphical representation of the screen template.
To generate media frames, information is supplied by choos
ing from a media frame catalog, or by typing an identifica
tion code to specify media data. For controls associated with
a screen, an operator supplies the text to define all buttons
and controls. For controls accessing one of the pre-defined
libraries, an operator assigns functionality to the control by
choosing from the function library catalog. Controls acti
vating a new screen are automatically assigned by the one to
one correspondence between the ordering of the template
and the structure within the interactive presentation editor.
Each control or media frame area located on a particular
screen template is selectable by the cursor control device. To
20
open a new or an existing screen project file. To open a new
or existing project file, a windows common dialog box is
invoked. The screen project files contain the file extension
“.SCR”. During the file load selection, only files containing
the file extension SCR are displayed for selection.
A “close" command closes the currently active screen
project file. The user is prompted to save changes if edits
were made subsequent to the last save operation. A "save”
command saves the active screen file to a permanent storage
device. A windows common dialog box is invoked in order
to save the screen project file. If no name is assigned to the
currently open screen project file, then the user is prompted
with a "save as” command. The "save as” command, also
15
20
select a control or media frame, the user double-clicks on the
area with the cursor control device, and an information box
is subsequently displayed by the screen editor program. For
a control, the information box displays the control text and
any attached functionality. For a media frame, the informa
tion box displays an associated identification. Although the
preferred manner of entering information is through the drag
and drop interface, information may be entered in the
information box directly by the user. However, for the
associated text relating to a particular control, the informa
tion box is the only means to enter data. In addition,
information is supplied in a read only type fashion for
defined controls in media frames. The defined controls are
preset within the template definition (e.g. an abort function).
Defined media frames are preset within the template
definition, normally for common text captions. The ability to
graphically select controls and or media frames on the
screen template provides an easy user interface to enter the
screen specific information. As screen specific information
is entered, the screen template is transformed so that the user
can evaluate the output display screen. The screen definition
maintains a reference to the screen template. Therefore,
when a screen template changes (e.g. a new logo is desired),
then, by definition, all screens using that particular screen
template as a reference are automatically updated.
FIG. 16 illustrates a GUI for the screen editor configured
in accordance with the present invention. In a preferred
embodiment, the screen editor program is implemented to
operate in conjunction with a windows operating system.
The screen editor program contains features such as a title
bar 2000, a menu bar 2010, a system menu button 2015, and
windows minimum/minimum buttons 2020. In addition,
other MicrosoftTM Window features are provided such as a
25
selectable from the file menu, saves the active project file
under a new name. A windows common dialog box is
invoked in order to save the active screen project file. A file
extension of SCR is automatically appended to the name
given by the user. The "Save As” command contains an
option in the file save dialog box to automatically save the
edits in all active child windows. Otherwise, the user is
prompted to save changes for each open child window.
A “New Screen” command opens a child window in order
to create a new screen for addition to the screen project file.
To open a new screen, the user supplies a valid screen
template name to associate a screen template for the new
screen. When the child window is closed, the user cancels
30
35
the edits or adds the screen to the "update list" used during
save operations. The "Open screen” command opens a child
window in order to edit an existing screen within the screen
set for the project file. A dialog box is invoked in order to
search the screen set for a particular screen. When opening
a new screen, the operator cancels the edits or adds the
screen to the update list. A "Print” command permits a user
to print the active screen. A windows common dialog box is
invoked to print the screen. A "Print Setup" command also
invokes a windows common dialog box in order set up the
printer environment. In addition, an “Exit” command per
mits the user to exit the screen editor program. If any updates
are made to screens currently active, the user is prompted to
save changes before exiting.
FIG. 17b illustrates an edit menu for the screen editor
program configured in accordance with the present inven
tion. The edit menu contains a number of commands to
manipulate screen sets in the screen editor program. An
45
"Undo' command allows the user to undo the last 'n' edit
operations. The number of reversible operations is a user
defined option that is set in the configuration program. In the
screen editor program, an "Undo” operation results in negat
ing information supplied for the control areas and media
50
frames. A "Delete” command allows the user to erase the
associated information for a control or media frame from the
database. A "Delete All" command, when selected, invokes
55
tool bar 2025, a status bar 2030 and scroll bars 2035 and
2037. The operation of the title bar, menu bar, system menu
button, tool bar, status bar, and scroll bars are described
above in conjunction with the interactive presentation editor.
FIG. 17a illustrates a file menu for the screen editor
configured in accordance with the present invention. The file
a dialog box to allow the user to delete all associated
information for a particular set of controls, media frames, or
any combination thereof. The delete all command is useful
for creating a new screen with the current screen template,
without having to remember the screen template name. The
"find” command permits a user to locate information asso
ciated with a control or media frame. For example, a user
may search based on the name of a media frame or the text
associated with a particular control on the screen.
menu for the screen editor contains a number of commands
FIG. 17c illustrates a view menu for the screen editor
to manipulate files within the screen editor program. A
program configured in accordance with the present inven
tion. In general, the view menu permits the user to select a
number of commands relating to the display of information
associated with the screen editor program. A "Zoom’ com
mand allows the user to set the area of display on the output
“new” command, selectable from the file menu, creates a
new screen project file. If a project file is currently opened,
the user is prompt to save any changes before An "open”
project file is created. An “open' command permits a user to
65
5,675,752
22
a language for the menu “LANGUAGE.API”; advance to
the next screen "NEXTSCR. API; return to the previous
21
display device. The user may either apply a Zoom scale
factor or select a window portion of the output display
device. If the user selects a window portion of the current
display, the selected window area is scaled to fit the window
display area. A “template mode” command permits a user to
place the screen editor program in the template mode so that
only the view of the screen template is shown. The template
mode does not display screens with the external media frame
files thereby allowing quick edits.
A "display mode” command places the screen editor
program in a display mode. The display mode permits a user
to update a view of the screen as information is supplied. For
example, when a media frame is assigned to an area on the
screen template, the actual media frame image is displayed
on the selected area. The display mode is the default mode
because it ensures to the operator that the correct data is
being supplied to the display screen. A "media frame”
catalog command toggles a media frame catalog “on” and
"off" the display screen. A check mark located beside the
media frame catalog command on the view menu indicates
the media frame catalog is displayed. In general, the media
frame catalog is a separate window that permits a user to
scroll through a list of media frames. The media frame
catalog assists a user by selecting a specific media frame,
and by placing the selected media frame on an area of the
25
is displayed along with a descriptive name. For the example
30
tation editor program. A screen template editor command
allows a user to access the screen template editor program.
Access to the screen editor template program from the
screen editor program permits the user to create/edit a screen
template while working on creation of the actual screens. A
"media frame editor” command permits access to the media
frame editor program. The media frame editor command
permits the user to create/edit a menu frame while working
35
operating mode for the screen editor program. The edit
mode, which is the normal mode of operation for the screen
editor program, permits the user to edit an active screen.
screen layout.
FIG. 18 illustrates an example of a media frame catalog
window configured in accordance with the present inven
tion. The graphics display portion 2400 of the media frame
screen “PREVSCR. API'; and abort a menu “ABORT.API.
FIG. 17.c illustrates a control function catalog command
selectable from the view menu. The control function catalog
command toggles the control function catalog “on” and
“off”. A check mark located beside the control function
O
to the tool bar command on the view menu indicates the tool
bar is displayed. Similarly, a status bar command toggles the
status bar on and off. A check mark next to the status bar
command on the view menu indicates that the status bar
15
FIG. 17d illustrates a tools menu for the screen editor
media frame. A select button 2410 is shown on the media
menu has a checkmark next to the command. The edit mode
from the test mode. The test mode command Switches the
operation mode of the screen editor from edit mode to a test
or demonstration mode. The testmode allows the user to test
the screen (i.e. run the media frame and use the controls) in
the manner intended for the end user. To return to the edit
mode, the user selects the edit mode command. A check
45
mark located next to the test mode command on the tools
menu indicates the test mode is the active mode,
50
FIG. 17e illustrates a help menu for the screen editor
program configured in accordance with the present inven
tion. The help menu contains the “help index” and “about”
commands. The help index command is configured in accor
dance with the standard WindowsTM help index type func
tion. In general, the help index function allows the user to
select from a category index to obtain help to operate the
55
screen editor program. An about command is implemented
as a standard WindowsTM type “about box” for a windows
program.
button, the screen editor program automatically creates an
The present invention includes a screen template editor to
provide capabilities for constructing screen template defini
tions. The screen template definitions include defining
backgrounds, borders, drop boxes, logos, controls, and
association between selection of the button and invocation
of the function. Note that controls that activate screens are
used to: purchase merchandice “PURCHASE.API'; reset
the menu to the original parent screen "RESETAPI'; select
When in the edit mode, the edit mode command on the tools
command allows a user to return the screen editor program
extensions “AP'.
automatically assigned functionality through use of the
interactive presentation editor program. Only controls per
forming other functions (e.g. handle credit card transactions)
are assigned using the function library catalog. However, the
function library catalog may be used to implement any
function. For example, the function library catalog may be
to create Screes.
An "edit' mode command allows a user to select an
description "Los Angeles Tourist drawing". In addition, the
media frame file "LA.MFR" identifies the file containing the
A select button 2405 permits a user to drag the icon box
2420 over a control button on the currently displayed screen.
If the user drags the icon and places the icon over a control
display is active.
configured in accordance with the present invention. The
commands selectable from the tools menu permit a user to
invoke different programs while maintaining the screen
editor program active. An interactive presentation editor
command permits the user to access the interactive presen
shown in FIG. 18, the media frame is associated with the text
frame catalog window. The select button 2410 is utilized to
drag a selected media frame icon over a media frame area on
the screen layout to create the association between the area
and the particular media frame selected.
The present invention includes a function library catalog.
The function library catalogis a separate window that allows
the user to scroll through a library of predefined functions.
The function library catalog permits the user to assign a
specific function to a control on the screen layout. FIG. 19
illustrates an example of a function library catalog window
configured in accordance with the present invention. Each
function contained within the function library has an asso
ciated text description and an icon box. In addition, the user
may scroll through the list of files contained in the function
library catalog for selection of the desired function. For the
example shown in FIG. 19, the "SHOPSUM.API file is
active, therefore the associated text description and icon box
are shown. The files for the function library are given the file
catalog command on the view menu indicates the functionis
active. A tool bar command on the view menu toggles the
tool bar display "on" and “off”. A checkmark located next
media frame windows. FIG. 20 illustrates a GUI for the
screen template editor configured in accordance with the
present invention. In a preferred embodiment of the present
invention, the screen template editor is implemented as a
65
Microsoft" Windows program. The screen template editor
screen layout contains a title bar 2600. The title bar 2600
includes a standard control/menu box 2601, 'screen tem
5,675,752
23
plate editor" title bar 2605, and a pair of minimize/maximize
buttons 2607. The screen template editor menu bar 2600
provides pull down access to menu selections. For example,
the screen template editor menu bar 2600 provides for
mouse click and accelerator key selection.
The screen template editor screen layout also contains a
tool bar 2610 for providing commonly used functionality to
the user. The tool bar 2610 includes buttons to pull up tool
dialogs for controls, logos, background and borders, and
media frame windows. A first button 2611 on the tool bar
2610 allows traversal ordering of controls for manual
assignment. A second button 2612 on the tool bar 2610
allows for testing the controls on a screen template to see
that a locked control user interface and pick access traversal
occur in the correct order. For purposes of explanation, a
locked control user interface is a user interface that does not
15
use a cursor, but instead provides some indication to the user
about the control currently ready for activation. Typically, a
set of cursor move buttons increments the particular indi
cation from one control to the next.
The screen template editor screen layout contains a plu
rality of tool dialog boxes. Specifically, the screen layout
contains a logos tool dialog box. 2615, backgrounds and
border tool dialog box. 2620, a media frame tool dialog 2625,
a controls tool dialog box 2630, and a screen template build
window 2635. All of the tool dialog boxes include a control/
menu button in the upper left corner. The control/menu
button is implemented from a standard MicrosoftTM Win
dows function. The control/menu button may be utilized to
close a tool dialog box. Once a tool dialog box has been
closed, the respective box is reopened by selecting the box
thumbnail sketch window 2628 allows a user to view the
corresponding highlighted selection in the list box. 2629. A
select button 2627 attaches a user's selection to the cursor
20
25
selection of a media frame window location and size within
30
box 2625 includes a list box. 2641 and a thumbnail sketch
35
box. 2642 for assigning specific media frames to the generic
screen template. The thumbnail sketch window 2642 also
shows sizing of the media frame window. A select button
2640 permits a user to select a media frame from the list box
2641. Selecting a media frame from the list box 2641
attaches the selected media frame to the cursor control
in accordance with the present invention. The controls tool
dialog box 2630 contains selections for selecting a control
type mode. Typical control type modes include buttons,
sliders and indicators. For the control tool dialog box
illustrated in FIG. 21a, the selections include buttons 2631,
sliders 2632, and indicators 2633. The buttons mode permits
a user to select from various types of buttons including
momentary, toggle, checkbox and radio button. The sliders
mode permits a user to select from controls with multiple
output states such as volume sliders and rotational knobs.
The indicators mode permits a user to select from various
sorts of light emitting diode (LED) indicators such as red,
yellow and green, circular LEDs, bargraph LEDs, semicircle
meter LEDs, and seven segment numerical readout LEDs.
The controls tool dialog contains a list box allowing control
selection from a set of bit map images. The selections,
located at the right hand side of controls tool dialog box
2630, provide the user with color, orientation, subtype, type
and attached function selection capabilities.
FIG. 21b illustrates a logos tool dialog box configured in
accordance with the present invention. The logos tool dialog
box 2615 allows users to select from sets of bit mapped
corporate logos. In a preferred embodiment, the corporate
logos are prescanned using an image editor such as AdobeTM
Photoshop. The corporate logos are loaded over a screen
templates background using an icon type graphics primitive
that supports a transparent surround for the logo. A list box
2617 provides descriptive labels for each corporate logo
currently selectable. Also, a thumbnail sketch window 2618
displays a picture of the highlighted logo description in the
list box. 2617. A select button 2619 attaches the selected
control device so that the user may drop the selected item
onto the screen template build window 2635. An additional
dialog box (not shown), invoked through a menu selection,
allows adding, deleting, and previewing new choices to the
backgrounds and borders tool dialog.
FIG. 21d illustrates a media frame windows tool dialog
box configured in accordance with the present invention.
The media frame windows tool dialog box 2625 allows
the generic screen template. The media frame tool dialog
from the tool bar 2610 or from a menu selection. In the
screen template editor, only one instance of each of the tool
dialogs can be invoked at one time. Opening multiple
instances of each tool dialog serves no useful purpose.
FIG. 21a illustrates a controls tool dialog box configured
24
corporate logo to the mouse cursor so that the logo may be
dropped into the screen template build window 2635. New
logo description labels are added to the logo tool dialog by
invoking a new logo dialog box (not shown). The new logo
dialog box permits adding, deleting, previewing, and can
celing of logos. In addition, the new dialog box permits entry
of text description for each logo.
FIG. 21c illustrates a backgrounds and borders tool dialog
box configured in accordance with the present invention.
The backgrounds and borders tool dialog box 2620 provides
users with options for controlling the graphical look of the
screen template. The backgrounds and borders tools dialog
box 2620 provides the user with capabilities to select
background color and texture, border color and style, and
various sorts of divider line and drop box shapes. A list box
2629 allows a user to scroll through possible selections. A
45
50
55
device. In addition, the selected media frame appears on the
output display with a dotted border comprising the size and
aspect ratio of the selected media frame. To select a media
for the screen template, the user places the dotted border
onto the generic screen template in the screen template build
window 2635, and drops the dotted border into place by
selecting a control button on the cursor control device. A
special selection within the list box 2641 allows the user to
retrieve a free formatted dotted border which can be adjusted
to any size or aspect ratio. The assign button 2643 attaches
specific media frames to the cursor control device for drag
and drop operations into the generic screen template on the
screen template build window 2635.
As shown in FIG. 20, the screen template build window
2635 provides a space for constructing a screen template. As
described above, the various component elements of a
screen template are attached to the mouse cursor and
dragged and dropped into the screen template build window
2635. In addition, selecting elements with the cursor control
device, already positioned within the screen template build
window, allows a user to easily reposition, cut, copy and
paste. Editing the screen template build window 2635 is
accomplished through the edit menu. Selecting, with the
cursor control device, the order button on the tool bar 2610
places the screen template into the order control traversal
mode. In the order control traversal mode, a user selects the
order in which a locked control user interface steps from one
control to the next.
65
A test button 2612, selectable from the tool bar 2610,
allows a user to test functionality attached to a particular
control. If the user selects the test button 2612, the screen
5,675,752
25
template editor enters the test mode. Upon invocation of the
test mode, a message box is displayed. The message box is
utilized to display a text string that indicates the nature of
operations attached to controls through the function button
within the controls tool dialog. As a user operates controls
in a screen template, any attached functionality is indicated
by an appropriate message. Abuild button returns the screen
template build window 2635 to the build mode.
When operating within the screen template editor, screen
templates are stored into a file. The file contains the name of
the active project and the file extension "STM". Any
number of screen templates are stored into the screen
26
depressing a button to select the symbol. The function
symbol is then dragged, with the button being depressed,
across a portion of a timeline track to generate a track edit,
Afade or dissolve begins when the mouse button is pressed,
and ends when the mouse button is released (as long as both
the mouse press and release occur within a single track
window). Also located on tool bar 2810 are three screen
O
template file, and are identified by screen template identi
fication values. An operations/screen template dialog menu
selection allows user access to screen template header
15
fields that permit a user to set a screen template's name,
description, id and reference to one other screen template.
When a screen template's identification value is edited, an
automatic adjustment of all other preassigned identification
values occurs. Consequently, each screen template identifi
cation is unique within a particular project. For example,
changing a screen template id to 20, whenid values of 20-25
are already defined within a particular project, automatically
bumps the preassigned ids to the range of 21-26. Screen
25
information. The operation/screen template dialog includes
templates are automatically assigned screen template iden
tification values in ascending order beginning at 1.
The screen template editor contains a reference capability
to allow a screen template to copy common characteristics
from another previously defined screen template. A next and
previous button, contained in a first screen template, may be
referenced by all other screen templates within the particular
project to automatically copy the control and logo charac
teristics. Specific characteristics referenced from another
screen template are displayed within the screen template
and a maximum time field 2822. The current time field 2821
30
vertical solid line cursor 2826 traverses the time scale and
the time scale units between seconds and minutes. The time
45
2830, 2832 and 2834. Each track contains a button label
50
placing the cursor control device on the function symbol and
sponding track, the button label displays the possible
choices indicating the current selection.
A slide bar 2850 at the bottom of the timeline window
55
2825 permits panning of the time scale and track displays in
the range from time zero up to the maximum time for the
media frame. A pair of bump up bump down buttons 2851
and 2852, respectively, permits a user to adjust the maxi
mum time for the media frame. The media frame editor of
The tool bar 2810 includes time line edit functions such
dissolve 2814. The timeline edit functions are selected by
units button 2829 label changes when the button is
depressed, toggling back and forth between "seconds” and
“minutes". The timeline window 2825 displays three tracks
2844 for Track 1, 2846 for Track 2, and 2848 for Track 3.
If a user successively presses a button label for a corre
box, "media frame editor' title, and minimize and maximize
buttons 2803. A menu bar 2805 provides pull down access
to menu selections. The menu bar 2805 provides for mouse
click and accelerator key entry selection.
as cut symbol 2811, fade in symbol. 2812, fade out 2813, and
media frame editor of the present invention. The timeline
window 2825 on the media frame editor screen display 2800
displays up to three media tracks against a time scale. A
the track window from left to right when a user runs a
particular media frame. Dotted vertical lines, such as dotted
vertical lines 2827 and 2828, indicate break point time
positions. A time units button 2829 permits a user to toggle
FIGS. 22a-d illustrate a GUI for the media frame editor
configured in accordance with the present invention. In a
preferred embodiment, the media frame editor is imple
mented as a MicrosoftTM Windows program. As shown in
FIG. 22a, the screen layout for the media frame editor
contains a title bar 2800, including a standard control/menu
displays the time position of the timeline cursor. The maxi
mum time field display 2822 indicates the total media frame
time length. Also shown on the toolbar 2810 is a compile
button 2823. The compile button 2823 permits periodic
saves under the currently active media frame file name. If no
media frame file name exists, the user is prompt by an input
dialog box so that the user may enter one.
FIG. 22b illustrates the timeline window 2825 for the
35
build window 2635. The referenced characteristics cannot
be repositioned or deleted. Instead, edits are performed on
the screen template where the controls and logos were
originally defined. The screen template editor also contains
a control order override so that controls copied from a
reference screen template may be reordered for control
traversal. The test mode for the screen template editor
permits controls and referenced screen templates to be tested
even though the assigned functionality cannot be altered.
Instead, assigned functionality of a reference screen tem
plate may be altered when the original reference screen
template is active in the screen template editor. The screen
template reference capability reduces network bandwidth in
file server storage requirements by sharing redundant screen
template definitions.
The media frame editor of the present invention provides
capabilities for combining images, graphics, animations,
video clips, text, and sound clips on a common timeline.
wipe buttons 2815, 1816, and 2817. The wipe buttons
provide, for example, horizontal, vertical, and diagonal
wipes between media elements. A wipe is a hard or soft
edged line transition from one media element to the next
media element. The media frame track edit syntax diagram
provides additional definitions of the on-screen edit marks.
The tool bar 2810 also provides buttons: set break point
2818, clear break point 2819, and clear all 2820. The set
breakpoint 2818, clear breakpoint 2819 and clear all 2820
set, clear, and clear all breakpoints, respectively. With the
breakpoint functions, multiple break points may be set by
the user. In operation, when a user sets a pair of breakpoints,
each successive run operation plays the selected tracks
between adjacent break points. If the user selects the clear
break point button 2819, the closest breakpoint to the
timeline cursor is cleared. The clear all break point button
2820 clears all currently set break points. The toolbar 2810
also displays information via a current time window 2821
65
the present invention utilizes standard film editing marks to
represent track cuts, fades, and dissolves. The track cuts,
fades, and dissolves define the length of media elements
within track window 2825. A cut is represented as a vertical
line in the track. Afade in is represented as a sideways “V”
pointing to the left as shown attime “0” on “Track 2". Afade
out is represented as a sideways "V" pointing to the right. A
dissolve between track elements is represented as a slanting
line from bottom to top of the respective track. Aslanting
line running from the top to bottom indicates a wipe. A
5,675,752
27
continuation of the current track elements is indicated by
two horizontal lines. A blank space in a particular track is
indicated by a single horizontal line in the center of the
corresponding track. The blank space indicates silence for
audio or blank screen for video. The length and position of
track editing marks are relative to the time scale. The media
frame track edit syntax diagram provides additional details
about track editing marks.
The media frame editor also contains a media selection
window 2852, located in the lower left corner of the media
frame editor screen display 2800, as illustrated in FIG. 22c.
28
FIG. 23a illustrates a file menu for the media frame editor
10
The media selection window 2852 allows selection of media
frames for attachment to a particular track. Within the media
selection window 2852, a plurality of buttons are provided
for selecting the type of media element. The buttons include
an image button 2836, graphics button 2837, animation
15
button 2838, video button 2839, sound button 2840 and text
button 2842. Alist box 2854 provides a selection mechanism
for specific media elements within the active media type,
and the active media element is shown at the top of the list
box 2854. A slide bar 2855 on the list box 2854 allows
traversal of the possible media selections.
A thumbnail sketch window 2859 provides a means to
show viewable media elements. For the example shown in
FIG. 22c, a video media element of New York is displayed
in the thumbnail sketch window 2859. A play button 2857
plays the currently selected media element. If the media
element is a sound clip, no view is shown in the thumbnail
sketch window 2859, but the corresponding sound clip is
played on the host computer system sound card. A select
20
25
30
button 2860 attaches the current media element to the cursor
control device thereby permitting a user to attach aparticular
media element to a portion of a track. The mouse cursor
changes to represent the type of media attached (e.g. a
musical note for sound, a rectangle for image, etc.). The
35
mouse cursor is then moved into a track window and clicked
on a portion of the track for which insertion is desired.
FIG. 22d illustrates a run window for the media frame
editor configured in accordance with the present invention.
The run window 2862 permits a user to play selected
portions of a media frame. A momentary run button 2864
plays the media frame between successive break points.
When the run button 2864 is selected, the cursor traverses
across the track window from left to right. A run view
window 2866 displays viewable elements of the media
frame, and any selected sound tracks are played on the
computer system sound card. An all button 2868 selects all
button, any of Tracks 1-3 can be deasserted.
When multiple viewable media elements overlap in time,
viewability priority is assigned automatically based on the
media type. In a preferred embodiment, animation contains
the highest viewability priority and is displayed over other
lower priority media element layers. The text media contains
the next highest priority layer, taking precedence over the
remaining media elements. The graphics is assigned the next
highest viewable priority, and an image has the lowest
viewable priority. The video clips are unique because video
clips are not combined with other viewable media image
elements. An attempt to lay down a video track in parallel
with any other viewable media element generates an error
message. The media frames are stored in files containing file
name extensions of '.MFR'.
associated file. The media frame file is saved under the name
"used' when the media frame was loaded. If a file name for
the media frame file was not previously specified, the user
is prompted to supply a file name. The media frame file
extension is automatically appended to a name supplied by
the user. Afile save as menu selection invokes a dialog box
to save the current program, sub-program or application
under a file name specified by the user. The standard media
frame file extension is automatically appended to the file
name designated by the user.
A "print” menu selection, selectable from the file menu,
invokes a dialog box to print the currently open media frame
file at the current timeline cursor position. The user specifies
an output format and number of copies for the print opera
tion. All selected viewable tracks are combined for printing,
and the layers for printing are the same as the layers for
viewing on the display screen. The "print setup" command,
when selected, invokes a common dialog box to set up a
printer. An "exit” menu selection doses active media frame
files. If revisions were made to the active open media frame
file subsequent to saving the file, then the user is prompted
to save edits prior to exiting the media frame program. If the
media frame editor was invoked from another portion of the
interactive applications generator program, control is
returned to the calling program.
45
FIG. 23b illustrates an edit menu for the media frame
editor configured in accordance with the present invention.
An “undo' command selection reverses the most recent
defined tracks for the current media frame. The buttons
2870. 2872 and 2874 are provided for selecting/deselecting
tracks 1-3, respectively. When the all button 2868 is
asserted, Tracks 1-3 buttons 2870, 2872 and 2874 are
automatically asserted. By selecting the associated track
configured in accordance with the present invention. The file
menu includes a new command. The new command, when
selected by a user, invokes a dialog box to create a new
media frame. If a media frame file is currently opened when
the new command is selected, the user is prompted to save
the currently opened mediaframe file prior to creating a new
media frame file. When operating in the interactive appli
cations generator environment, only one media frame may
be active at any one time. An "open' command, when
selected by a user, invokes a dialog box with a list box to
select an existing media frame. All media frame files con
taining the file extensions “MFR" are displayed in a combo
box within the file open dialog.
A "close” command, selectable from the file menu, closes
a currently opened media frame file. If edits were made to
the active media frame file, the user is prompted to save
changes. A "save” menu selection, when invoked, saves the
currently open program, Sub-program, or application to the
50
55
track edit operation performed by the user. A “cut” menu
selection deletes the selected portions of a track. The deleted
portion of the trackis copied to the windows clip board, and
is accessible to the window track at a future time. A "copy”
menu selection copies selected portions of a track to the
windows clip board. A “paste” menu selection inserts the
contents of the windows clipboard into the track window at
a point designated by a user. The "delete” menu selection
removes the currently selected portion of a track.
FIG. 23c illustrates a view menu for the media frame
editor configured in accordance with the present invention.
A "zoom in” command, selectable from the view menu,
permits the user to decrease the displayed time scale range
65
down to 75% of the previously displayed time range. The
"zoom out' view menu selection allows the user to increase
the time scale displayed range up to 133% of the currently
displayed time range. A "seconds” and “minutes”
commands, selectable from the view menu, permits the user
to toggle between the seconds and minutes time scale. When
invoked by the user, the menu frame editor redraws the time
5,675,752
29
scale with the appropriate units. In addition, the media frame
editor changes the current and maximum time windows
2821 and 2822, respectively to reflect the selection. A "track
set' view menu selection allows a user to pull up a menu box
to save the three currently selected tracks as either track set
1, 2, 3 or 4. The dialog box also allows selection of any of
the four possible track sets as the current set.
FIG. 23d illustrates an operations menu for the media
frame editor configured in accordance with the present
invention. A “cut” operations menu selection permits a user
to attach a track cut edit operation to the cursor control
device. A track cut edit operation provides a sharp cut from
one media element to the next subsequent media element.
FIG. 24a illustrates the media element track cut edit syntax
for a preferred embodiment of the present invention. The
media element track cut edit syntax is displayed in the tool
bar 2810 as cut symbol 2811. In a preferred embodiment, the
track cut editis applied to a track when the leftmouse button
is pressed and released.
10
15
The “fade in” command, selectable from the operations
menu illustrated in FIG. 23d, permits a user to attacha track
fade in edit operation to the cursor control device. FIG. 24b
illustrates the media element track fade in edit syntax for a
preferred embodiment of the present invention. The fade in
edit operation provides a gradual fade in from a blank into
25
the media element. The media element track fade in edit
syntax is displayed in the tool bar 2810 as fade in symbol
2812. In the preferred embodiment, the fade in edit opera
tion is applied to a track by pressing a left button on the
cursor control device at the start of the fade in, dragging the
mouse cursor to the end of the fade in, and releasing the
mouse button. In order to effectively apply the edit opera
tions to a track, the mouse cursor begins and ends within the
same track window.
A "fade out” menu selection on the operations menu
permits a user to attach a trackfade out edit operation to the
30
operation provides a gradual transition to mix two media
mouse cursor. In a preferred embodiment, FIG. 24g illus
trates the media element track horizontal wipe edit syntax
from top to bottom, and FIG. 24h illustrates the media
element track horizontal wipe edit syntax from bottom to
top. The media element track vertical wipe edit syntax is
displayed in the tool bar 2810 as screen wipe symbol 2816.
The currently selected horizontal wipe is applied to a track
by pressing the left mouse button at the start of the wipe,
dragging the mouse cursor to the end of the wipe, and
releasing the mouse button. The mouse cursor begins and
ends within the same track window for the wipe to be
applied to a particular track.
The "diagonal wipe' selection on the operation menu
permits a user to attach a track wipe edit operation to the
mouse cursor. In a preferred embodiment, FIG. 24i illus
trates the media element track diagonal wipe edit syntax
35
45
50
55
elements between the transition from the first to the second.
The media element track dissolve edit syntax is displayed in
the tool bar 2810 as dissolve symbol 2814. The dissolve edit
operation is applied to a track by pressing the left mouse
button at the start of the dissolve, dragging the mouse cursor
to the end of the dissolve and releasing the mouse button.
The mouse cursor begins and ends within the same track
window for the dissolve operation to effectively operate on
illustrates the media element track diagonal wipe edit syntax
from upper left to lower right corner. In addition, FIG. 24k
illustrates the media element track diagonal wipe edit syntax
from lower right to upper left hand corner, and FIG. 24l
illustrates the media element track diagonal wipe edit syntax
from upper right to lower left corner. The media element
track vertical wipe edit syntax is displayed in the tool bar
2810 as screen wipe symbol 2817. The currently selected
diagonal wipe direction is used. The "diagonal wipe direc
tion” selection on the operations menu invokes a dialog box,
when selected by a user, that allows the user to select a
diagonal wipe direction. The diagonal wipe directions
include: upper right to lower left corners, lower right to
upper left corners, upper left to lower right corners, and
lower left to upper right comers.
A "set breakpoint” menu selection on the operations
menu, illustrated in FIG. 23d, permits a user to attach a break
point to the mouse cursor. A breakpoint is set by the userby
pressing a left mouse button and releasing the button within
the time scale window, or within any of the three track
windows. A "clear breakpoint" menu selection on the opera
tions menu permits a user to clear the breakpoint closest to
the current cursor position. If no break points are currently
set, a message box displays the message “no breakpoints are
currently set.” After displaying the message box, the menu
frame editor waits for the user to press an "OK" button. A
“clear all breakpoints' menu selection on the operations
menu permits a user to clear all currently set break points.
If no breakpoints are set, the media frame editor displays a
message box containing the message, “no break points are
currently set.” In response, the user is prompted to press an
a track.
A“vertical wipe' menu selection permits a user to attach
a track wipe edit operation to the mouse cursor. In a
preferred embodiment, FIG. 24e illustrates the media ele
from left to right.
The "horizontal wipe" selection on the operation menu
permits a user to attach a track wipe edit operation to the
from lower left to upper right hand corner, and FIG. 24.j
mouse cursor. FIG. 24c illustrates the media element track
fade out edit syntax for a preferred embodiment of the
present invention. The fade out edit syntax provides a
gradual fade outfrom a media element to a blank track. The
media element trackfade out edit syntax is displayed in the
tool bar 2810 as fade out symbol 2813. In a preferred
embodiment, the fade out edit operation is applied to a track
by pressing the leftmouse button at the start of the fade out,
dragging the mouse cursor to the end of the fade out, and
releasing the mouse button. The mouse cursor begins and
ends within the same track window to effectively execute the
fade out operation.
A "dissolve” menu selection, selectable from the opera
tions menu, permits a user to attach a track dissolve edit
operation to the mouse cursor. FIG. 24d illustrates the media
element track dissolve edit syntax for a preferred embodi
ment of the present invention. The dissolve track edit
30
ment track vertical wipe edit syntax from left to right, and
FIG. 24f illustrates the media element track vertical wipe
edit syntax from right to left. The media element track
vertical wipe edit syntax is displayed in the toolbar 2810 as
screen wipe symbol 2815. The currently selected vertical
wipe is applied to a track by pressing the left mouse button
at the start of the wipe, dragging the mouse cursor to the end
of the wipe, and releasing the mouse button. In order to
effectively apply the wipe operation to a track, the mouse
cursor begins and ends within the same track window. The
"vertical wipe direction” selection on the operations menu,
upon selection by a user, invokes a dialog box allowing the
user to select a vertical wipe direction from right to left, or
“OK” button. A “set maximum time” command selectable
65
from the operations menu invokes a dialog box when
selected by a user that allows the user to set the maximum
time of the media frame.
5,675,752
31
32
FIG. 23e illustrates a tools menu for the media frame
frame focuses on the center two balloons and the Zoom
editor configured in accordance with the present invention.
An "image editor” menu selection on the tools menu permits
a user to call the selected image editor. An image editor may
comprise any commercially available multimedia editing
program. A 'graphics editor” menu selection on the tools
menu calls the selected graphics editor when invoked by the
display effect opens up to shown all five hot air balloons.
The zoom display effect of the present invention provides a
user with the Zoom rectangle timing by graphing the
acceleration, velocity, and size change as the media frames
Zoom from the first key frame to the second key frame.
FIG. 27 illustrates a first complex pan and zoom effect for
the media frame editor of the present invention. For the first
complex pan and zoom effect illustrated in FIG. 27, four key
frames are specified. The first key frame, showing the torch
of the Statue of Liberty, and the second key frame shows the
upper portion of the statue. The third key frame shows most
of the statue, and the fourth key frame shows the entire
media frame available. Consequently, the resulting media
frames both pan and zoom the Statue of Liberty and the New
York city background. For the complex pan and zoom effect,
the media frame editor provides Zoom rectangle image and
path movement timing information. The key frames are
designated by an oval on the timeline, and the intermediate
pull points are designated by a square. For the complex pan
and zoom effect illustrated in FIG. 27, four intermediate pull
points are designated. As discussed above, the Zoom rect
angle image and path movement timing show the
acceleration, velocity and size change for the respective
display effect,
FIG. 28 illustrates a second complex pan and zoom effect
for the media frame editor of the present invention. The
second complex pan and zoom effect includes two key
frames. As shown in the display window, the first key frame
shows a portion of the hot air balloon in the foreground, and
the second key frame includes the entire media frame
subject matter. For the second complex pan and Zoom effect
illustrated in FIG. 28, five intermediate pull points are
specified. Consequently, in seven frames, the media frame
editor scans the media frame subject matter from the first
key frame to the second key frame. As described above, the
media frame editor provides Zoom rectangle image and path
movement timing.
FIGS. 29a-fillustrate a target application media frame
format configured in accordance with the present invention.
user. The "animation editor” menu selection on the tools
menu calls an animation editor when selected by the user. A
'video editor” menu selection and a "sound editor” menu
selections, selectable from the tools menu, calls a video and
sound editor, respectively. Similarly, a "text editor” menu
selection on the tools menu calls a text editor. A "pick
editors' menu selection on the tools menu invokes a dialog
box allowing selection of various media element editors. In
a preferred embodiment, the dialog box contains a set of
standard industry programs commercially available. In
10
15
addition, the pick editors menu selection permits a user to
specify a program name and search path for the program
name. A "run selected' menu command selectable from the
tools menu plays selected tracks of media frames between
adjacent break points. A “run all” menu selection, when
invoked, plays all tracks of the media frames between
adjacent break points. A "compile media frame” menu
Selection, selectable from the tools menu, compiles tracks of
a media frame into a menu frame file with the file extension
“MFR.
20
25
FIG. 23fillustrates a help menu for the media frame editor
configured in accordance with the present invention. A "help
index” menu selection, when selected by a user, invokes a
help dialog with a list of help entries for the media frame
editor. An “about' menu selection on the help menu invokes
an about dialog box that lists the current version of the media
frame editor and the release date. The dialog box also lists
Sony Trans Com, Inc. address and ownership message.
FIG.25 illustrates a pan display effect for the media frame
editor for the present invention. In general, the pan display
effect permits a user of the interactive applications generator
to pan media subject matter for presentation in the interac
tive presentation environment. In order to generate the pan
display effect, the media frame editor provides intermediate
pull points and key frames. For the example illustrated in
FIG. 25, the media subject matter contains seven interme
diate pull points and two key frames. The key frames define
a beginning frame and an ending frame for the pan display
effect. The intermediate pull points define intermediate or
intermittent points between the beginning and ending key
frames. In order to completely define the pan display effect,
the media frame editor provides path movement timing. The
path movement timing includes accelerator, velocity, and
size change. The acceleration and velocity define the rate at
which frames, representing the intermediate pull points, are
displayed. For the example illustrated in FIG. 25, the city
scape is displayed in nine frames, beginning with the first
key frame, shown in the left portion, and continuing to the
second key frame shown in the right portion of the display
30
35
40
As shown in FIG. 29a, the media frame format includes a
45
mation offset, and video offsetfields. FIG. 29billustrates the
sound offset field containing a sound item count field. In
addition, for each sound item, the sound offset field contains
50
55
window. The media frame editor also includes a disable
Zoom, disable frame, a show move and play move controls.
In addition, on the bottom portion of the media frame editor
display, the time elapsed for the pan display effect and the
amount of Zoom are shown.
FIG. 26 illustrates a zoom display effect for the media
frame editor of the present invention. Analogous to the pan
display effect shown in FIG. 25, the zoom display effect
involves specifying key frames and intermediate pull points.
For the example illustrated in FIG. 26, two key frames and
seven intermediate pull points are defined. The first key
media frame id, media frame size, and media types count.
The media types count specifies a length for the various
multimedia types. Specifically, the media frame format
contains a sound offset, image offset, graphics offset, ani
65
a sound item type, sound item offset, sound item size, and
sound item time fields. The sound item offset field specifies
an address for the corresponding sound item data. FIG. 29c
illustrates the image offset field. The image offset field
specifies an image item count to specify the length of the
image item fields. For each image item, a type, offset, size
and time fields are specified. The offset field specifies the
starting location for the corresponding image item data.
Similarly, the graphics, animation, and video multimedia
types are specified as shown in FIGS. 29d, 29e, and 29f,
respectively.
FIG. 30 illustrates a generalized output format for the
interactive applications generator of the present invention.
In a preferred embodiment, the interactive applications
generator stores an intermediate data format generated by
each of the primary component parts. The output data format
is optimized for modification and flexible editing. In addi
tion to the primary components described above, the present
invention operates in conjunction with an output program
5.675,752
33
generator. The output program generator creates file server
and user portions of code and media descriptions for a
particular interactive presentation environment. The output
format is application specific, and therefore varies, depend
ing upon the target interactive application environment. As
is well known in the art, multiple target output formats may
be created from a single intermediate output format.
Typically, the target output formats are optimized for per
formance and size. For example, target output formats may
be compatible with Microsoft(s) modular windows, Kalei
da's Script X, or other typical multimedia scripting or
graphical user interface languages. In addition, as one
skilled in the art will appreciate, other application specific
formats may be generated from the intermediate format of
the present invention.
The interactive applications generator of the present
invention may be implemented with software to perform the
functions of the screen template editor, the media frame
editor, the screen editor, and the interactive presentation
editor. In one embodiment, the interactive applications gen
erator is implemented with object-oriented computer pro
grams. FIGS. 31a-billustrate, in Booch notation, one imple
mentation for the interactive applications generator.
However, any programming language may be used to imple
ment the interactive applications generator without deviat
ing from the spirit and scope of the invention.
Although the present invention has been described in
terms of a preferred embodiment, it will be appreciated that
various modifications and alterations might be made by
those skilled in the art without departing from the spirit and
scope of the invention. The invention should therefore be
34
4. The apparatus for generating an interactive applications
environment as claimed in claim 3, wherein said controls
dialog box on said screen template editor GUI permits
selection of buttons, sliders and indicators.
5. The apparatus for generating an interactive applications
environment as claimed in claim 4, wherein said buttons
comprise momentary, toggle, checkbox and radio button.
6. The apparatus for generating an interactive applications
environment as claimed in claim 4, wherein said sliders
10
environment as claimed in claim 4, wherein said indicators
comprise red light emitting diode (LED) indicators, yellow
(LED) indicators, green (LED) indicators, circular LEDs,
bargraph LEDs, semicircle meter LEDs, and seven segment
numerical readout LEDs.
8.The apparatus for generating an interactive applications
environment as claimed in claim3, wherein said background
and borders dialog box on said screen template editor GUI
20
25
30
40
45
50
tions environment as claimed in claim 1, wherein said media
tions environment as claimed in claim 1, wherein said media
frame editor comprises a media frame editor graphical user
interface (GUI) for displaying a timeline window compris
ing a plurality of media tracks, a time scale and a vertical
CSO.
55
environment as claimed in claim 1, wherein said screen
template editor comprises a screen template editor graphical
user interface (GUI) for displaying a plurality of tool dialog
boxes to permit a user to select said plurality of generic
screen template elements, and for displaying a screen tem
plate build window for designing said screen template.
3. The apparatus for generating an interactive applications
environment as claimed in claim 2, wherein said plurality of
tool dialog boxes within said screen template editor GUI
comprise a logos dialog box, a backgrounds and borders
dialog box, controls dialog box and media frame window
dialog box.
frame windows dialog box on said screen template editor
GUI comprises a list box, for listing a plurality of media
frames currently selectable, a thumbnail sketch window for
illustrating specific media frames, including sizing, of a
media frame highlighted in said list box, and a select button
to permit a user to select a media frame window location
within the generic screen template, such that a dotted border
comprising size and aspect ratio of said media frame
selected is displayed in said screen template build window.
11. The apparatus for generating an interactive applica
frames generated by said media frame editor comprise
images, graphics, animations, video clips, text, and sound
clips.
12. The apparatus for generating an interactive applica
environment.
2. The apparatus for generating an interactive applications
environment as claimed in claim 3, wherein said logos
dialog box on said screen template editor GUI comprises a
list box, for listing a plurality of logos currently selectable,
a thumbnail sketch window for displaying a picture of a
highlighted logo in said list box, and a select button to permit
tions environment as claimed in claim3, wherein said media
35
elements;
a media frame editor for generating media frames by
combining multimedia component elements in a time
synchronized manner;
a screen editor for generating display screens, including
assigning functionality to said display screens, for use
in said interactive applications environment, said
screen editor being constructed to reference generic
screen templates and media frames to create a unique
instance for each display screen in said interactive
applications environment; and
an interactive presentation editor for creating hierarchical
structures for presentation of said display screens,
wherein said hierarchical structure defines presentation
of said display screens in said interactive presentation
comprises a list box, for providing options to define a
graphical look of said screen template, and a thumbnail
sketch window for displaying a corresponding highlighted
selection in said list box, and a select button to permit a user
to specify background color, background texture, border
color, border style, drop boxes and lines.
9. The apparatus for generating an interactive applications
a user to select from said plurality of logos.
10. The apparatus for generating an interactive applica
measured in terms of the claims which follow.
What is claimed is:
1. An apparatus for generating an interactive applications
environment, said apparatus comprising:
a screen template editor for creating generic screen tem
plates comprising a plurality of generic screen template
comprise volume sliders and rotational knobs.
7. The apparatus for generating an interactive applications
13. The apparatus for generating an interactive applica
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a plurality of time
line edit functions including a cut, fade in, fade out, dissolve,
horizontal wipe, vertical wipe, and digital wipe.
14. The apparatus for generating an interactive applica
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a slide bar within
said timeline window to permit a user to pan the time scale
and media tracks.
65
15. The apparatus for generating an interactive applica
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a media selection
5,675,752
35
window, including a list box and a plurality of buttons, to
permit a user to attach media frames to one of said plurality
36
plurality of screen identifier icons displayed on an output
display, wherein said plurality of screen identifiers corre
spond to a plurality of display screens, said interactive
of media tracks.
presentation editor GUI being constructed to permit place
ment of said plurality of screen identifiers in a hierarchical
Structure.
27. The apparatus for generating an interactive applica
16. The apparatus for generating an interactive applica
tions environment as claimed in claim 15, wherein said
media selection window comprises a thumbnail sketch win
dow for displaying viewable media elements selected via
tions environment as claimed in claim 26, wherein said
interactive presentation editor graphical user interface (GUI)
said list box.
17. The apparatus for generating an interactive applica
tions environment as claimed in claim 12, wherein said
10
media frame editor GUI further comprises a run operation
and a run window to permit a user to view selected media
frames.
18. The apparatus for generating an interactive applica
tions environment as claimed in claim 17, wherein said
15
media frame editor GUI further comprises a plurality of
breakpoints to play successive media frames on a selected
media track between adjacent breakpoints on said run win
dow.
19. The apparatus for generating an interactive applica
20
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a pan function to
generate a series of media frames that pans across a media
element.
20. The apparatus for generating an interactive applica
25
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a zoom function
to generate a series of media frames that zooms in and out
structure; and
30
tions environment as claimed in claim 12, wherein said
media frame editor GUI further comprises a pan and zoom
function to generate a series of media frames that zooms in
and out and pans across a media element.
22. The apparatus for generating an interactive applica
being a drag and drop type operation permitting place
ment of a corresponding screen identifier, representing
a menu screen selected, in any level of said hierarchical
of a media element.
21. The apparatus for generating an interactive applica
comprises a plurality of dialog boxes corresponding to said
plurality of display screens to permit entry of specific
display screen information, linking said corresponding dis
play screen to a media frame and a screen template, a list of
each control on said corresponding display screen, and a link
to either an application screen or menu screen.
28. The apparatus for generating an interactive applica
tions environment as claimed in claim 27, wherein said
specific display Screen information entered through said
plurality of dialog boxes further comprises grouping infor
mation to permit group level classification of display
SCS
29. The apparatus for generating an interactive applica
tions environment as claimed in claim 26, wherein said
interactive presentation editor GUI comprises:
a menu screen operation to add a new menu screen to said
hierarchical structure, said menu screen operation
35
tions environment as claimed in claim 1, wherein said
an application screen operation to add a new application
screen to said hierarchical structure, said application
screen operation being a drag and drop type operation
permitting placement of a screen identifier, represent
ing a application screen selected, in any level of said
hierarchical structure.
30. The apparatus for generating an interactive applica
display Screens comprise menu screens and application
screens, said menu screens being constructed to permit
Selection of another menu screen or an application screen,
and said application screen comprising a graphical user
interface (GUI) for a particular application program oper
ating within said interactive presentation environment.
23. The apparatus for generating an interactive applica
tions environment as claimed in claim 29, wherein said
tions environment as claimed in claim 1, wherein said screen
from said parent screen to said menu screen or said appli
cation screen selected.
31. The apparatus for generating an interactive applica
tions environment as claimed in claim 29, wherein said
application screen operation comprises an application group
editor comprises a screen editor graphical user interface
(GUI) for displaying a screen layout including a graphical
representation of said generic screen template referenced by
said screen editor, and comprising media frame windows
and controls.
24. The apparatus for generating an interactive applica
menu screen operation and said application screen operation
comprises an automatic forward link to create a forward link
from a parent menu screen to said selected menu screen and
application screen, said automatic forward link operation
being invoked upon placement of said screen identifier over
a parent menu screen identifier to establish a forward link
45
function to permit grouping of particular application
50
tions environment as claimed in claim.1, wherein said screen
editor further comprises a media frame catalogue compris
ing a separate window and being constructed to permit a user
to scroll through a catalogue of media frames so as to assign,
via a drag and drop operation, a media frame to a media
frame area on said screen layout.
25. The apparatus for generating an interactive applica
55
editor further comprises a function library catalog compris
ing a separate window and being constructed to permit a user
to scroll through a library of predefined functions so as to
assign, via a drag and drop operation, a specific function to
a control on said screen layout.
26. The apparatus for generating an interactive applica
tions environment as claimed in claim 1, wherein said
tions environment as claimed in claim 26, wherein said
interactive presentation editor GUI further comprises a
screen catalog comprising a window and being constructed
to permit a user to scroll through a library of display screens
so as to select a display screen for placement in said
tions environment as claimed in claim 1, wherein said screen
interactive presentation editor comprises an interactive pre
sentation editor graphical user interface (GUI) comprising a
screens, said interactive presentation editor GUIbeing con
structed to display said application screens in a horizontal
direction to define an order for presentation of said appli
cation screens in said interactive presentation environment.
32. The apparatus for generating an interactive applica
interactive applications environment.
33. The apparatus for generating an interactive applica
tions environment as claimed in claim 26, wherein said
65
interactive presentation editor GUI further comprises a
forward path command to establish a link from a control in
a parent menu screen to a child display screen.
34. The apparatus for generating an interactive applica
tions environment as claimed in claim 26, wherein said
5,675,752
37
interactive presentation editor GUI comprises a back path
command to establish link from a child display screen to exit
to a screen other than a corresponding parent screen.
35. A method for generating an interactive applications
environment, said method comprising the steps of:
providing a plurality of multimedia component elements;
generating at least one generic screen template compris
ing a plurality of generic screen template elements;
generating media frames by combining said multimedia
component elements in a time synchronized manner;
generating display screens for use in said interactive
applications environment, by referencing said generic
screen templates and said media flames to create a
unique instance for each display screen in said inter
active applications environment;
assigning functionality to said display screens; and
generating hierarchical structures for presentation of said
display screens, wherein said hierarchical structure
defines presentation of said display screens in said
interactive presentation environment.
36. The method as set forth in claim 35, wherein the step
of generating at least one generic screen template comprises
the steps of:
displaying a plurality of tool dialog boxes to permit a user
to select said plurality of generic screen template
38
or displaying a dotted border comprising size and aspect
ratio of said media frame selected.
41. The method as set forth in claim 35, wherein the step
of providing a plurality of multimedia component elements
comprises the step of providing images, graphics,
animations, video clips, text, and sound clips.
42. The method as set forth in claim 35, wherein the step
of generating media frames comprises the step of generating
a timeline window comprising a plurality of media tracks, a
time scale and a vertical cursor,
5
25
elements; and
displaying a screen template build window for designing
said screen template.
37. The method as set forth in claim 36, wherein the step
of displaying a plurality of tool dialog boxes comprises the
steps of:
displaying a logos dialog box;
displaying a background and borders dialog box;
displaying a controls dialog box; and
displaying a media frame window dialog box.
38. The method as set forth in claim 37, wherein the step
of displaying a background and borders dialog box com
30
picture of a highlighted logo in said list box; and
generating a select button to permit a user to select from
said plurality of logos.
35
45
48. The method as set forth in claim 47, wherein the step
of generating a timeline window comprises the step of
generating a plurality of breakpoints to play successive
media frames on a selected media track between adjacent
breakpoints on said run window.
49. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a pan function to generate a series of media
frames that pans across a media element.
50. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a Zoom function to generate a series of media
frames that Zooms in and out of a media element.
51. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a pan and zoom function to generate a series of
media frames that Zooms in and out and pans across a media
50
element.
52. The method as set forth in claim 35, wherein the step
of generating display screens comprises the steps of:
displaying menu screens to permit selection of another
55
40. The method as set forth in claim 37, wherein the step
of displaying a media frame window dialog box comprises
menu screen or an application screen; and
generating application screens to provide a graphical user
interface (GUI) for a particular application program
operating within said interactive presentation environ
ment.
the steps of:
displaying a list box of a plurality of media frames
currently selectable;
displaying a thumbnail sketch window for illustrating
specific media frames, including sizing, of a media
frame highlighted in said list box;
generating a select button to permit a user to select a
media frame window location within the generic screen
template; and
47. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a run operation and a run window to permit a user
to view selected media frames.
displaying a list box that provides options to define a
graphical look of said screen template;
generating a thumbnail sketch window that displays a
corresponding highlighted selection in said list box;
and
frames to one of said plurality of media tracks.
46. The method as set forth in claim 45, wherein the step
of generating a media selection window comprises the step
of generating a thumbnail sketch window for displaying
viewable media elements selected via said list box.
prises the steps of:
displaying a select button to permit a user to specify
background color, background texture, border color,
border style, drop boxes and lines.
39. The method as set forth in claim 37, wherein the step
of displaying a logos dialog box comprises the steps of:
displaying a plurality of currently selectable logos;
displaying a thumbnail sketch window that displays a
43. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a plurality of time line edit functions including a
cut, fade in, fade out, dissolve, horizontal wipe, vertical
wipe, and digital wipe.
44. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a slide bar within said timeline window to permit
a user to pan the time scale and media tracks.
45. The method as set forth in claim 42, wherein the step
of generating a timeline window comprises the step of
generating a media selection window, including a list box
and a plurality of buttons, to permit a user to attach media
53. The method as set forth in claim 35, wherein the step
of generating display screens comprises the step of display
ing a screen layout including a graphical representation of
said generic screen template referenced comprising media
frame windows and controls.
65
54. The method as set forth in claim 35, wherein the step
of generating display screens comprises the step of gener
ating a media frame catalogue comprising a separate win
dow to permit a user to scroll through a catalogue of media
5,675.752
40
operation permitting placement of a screen identifier,
representing an application screen selected, in any level
39
flames so as to assign, via a drag and drop operation, a media
frame to a media frame area on said screen layout.
of said hierarchical structure,
55. The method as set forth in claim 35, wherein the step
of generating display screens comprises the step of gener
60. The method as set forth in claim.59, wherein the steps
ating a function library catalog comprising a separate win 5 of generating a menu screen operation and generating an
dow to permit a user to scroll through a library of predefined application screen operation comprises the steps of:
functions so as to assign, via a drag and drop operation, a
generating an automatic forward link to create a forward
specific function to a control on said screen layout.
link from a parent menu screen to said selected menu
56. The method as set forth in claim 35, wherein the step
screen and application screen; and
of generating hierarchical structures comprises the step of 10 invoking, upon placement of said screen identifier over a
generating a plurality of screen identifier icons on an output
parent menu screen identifier, said automatic forward
display, wherein said screen identifiers correspond to said
link operation to establish a forward link from said
display screens to permit placement of said plurality of
parent screen to said menu screen or said application
screen identifiers in said hierarchical structure.
57. The method as set forth in claim 56, wherein the step
of generating a plurality of screen identifier icons comprises
the steps of:
displaying a plurality of dialog boxes corresponding to
said display screens to permit entry of specific display
screen information including text description of a cor
responding display screen;
permitting a user to link said corresponding display screen
to a media frame and a screen template;
displaying a list of each control on said corresponding
display screen; and
permitting a user to link said control to either an appli
15
25
cation screen or menu screen.
58, The method as set forth in claim 57, further compris
ing the step of permitting a user to group information to
classify group levels of display screens.
59. The method as set forth in claim 56, wherein the step
of generating a plurality of screen identifier icons on an
output display comprises the steps of:
generating a menu screen operation to add a new menu
30
of generating hierarchical structures further comprises the
35
screen to said hierarchical structure, said menu screen
said hierarchical structure; and
step of generating a forward path command to establish a
link from a control in a parent menu screen to a child display
SCC
operation being a drag and drop type operation permit
ting placement of a corresponding screen identifier,
representing a menu screen selected, in any level of
generating an application screen operation to add a new
application screen to said hierarchical structure, said
application screen operation being a drag and drop type
screen selected.
61. The method as set forth in claim 59, wherein the step
of generating a application screen operation comprises the
steps of:
generating an application group function to permit group
ing of particular application screens; and
displaying said application screens, based on said appli
cation group function, in a horizontal direction to
define an order for presentation of said application
screens in said interactive presentation environment.
62. The method as set forth in claim 56, wherein the step
of generating hierarchical structures comprises the steps of:
generating a screen catalog comprising a window and a
library of display screens; and
permitting a user to scroll through said library of display
screens so as to select a display screen for placement in
said interactive applications environment.
63. The method as set forth in claim 56, wherein the step
40
64. The method as set forth in claim 56, wherein the step
of generating hierarchical structures further comprises the
step of generating a back path command to establish a link
from a child display screen to exit to a screen other than a
corresponding parent Screen.
UNITED STATES PATENT AND TRADEMARK OFFICE
CERTIFICATE OF CORRECTION
PATENT NO.
5,675,752
DATED
:
October 7, 1997
INVENTOR(S) :
Scott et al.
It is certified that error appears in the above-identified patent and that said Letters
Patent is hereby corrected as shown below:
in Column 3 at line 34 delete "flames' and insert
--frames--
in column 9 at line 16 delete "flames" and insert
--frames--
In Column 30 at line 47 delete "comers' and insert
--Corners--
In Column 37 at line 13 delete "flames" and insert
--frames--
in Column 38 at line 1 delete "or"
In Column 39 at line 1 delete "flames' and insert
--frames--
In column 40 at line 16 delete "a application" and insert
--an application--
Signed and Sealed this
Sixteenth Day of November, 1999
2.72%Q. TODD DICKINSON
Attesting Officer
Acting Commissioner of Paten is and Trade not rks
Download PDF
Similar pages