2206a 2206b 2206C
US006738075B1
(12)
(54)
(75)
United States Patent
(10) Patent N0.:
Torres et al.
(45) Date of Patent:
6,072,480 A *
6,215,523 B1 *
6/2000 Gorbet et a1. ............. .. 345/302
4/2001 Anderson
.... .. 348/3335
DIGITAL IMAGING DEVICE
6,237,010 B1 *
5/2001 Hui et al.
...... .. 707/502
6,249,316 B1 *
6/2001
.... .. 348/3335
Inventors: Dan Torres, San Carlos, CA (Us); Eric
_
_
Notice:
_
_
(57)
_
_
_
_
_
_
_
show in a digital imaging device. The method aspect of the
present invention includes ?rst creating a slide shoW from
randomly selected ones of the heterogeneous media objects
stored in the digital imaging device, Where each one of the
(21) Appl- NO-I 09/223,961
(22) Filed
Dec 31 1998
'
ABSTRACT
A method and apparatus for interactively presenting a shde
Thl_s Patent 15 Sublect to a termmal dls'
Clalmer'
'
_
cited by examiner
(74) Attorney, Agent, or Firm—SaWyer LaW Group LLP
U.S.C. 154(b) by 0 days.
_
_
Primary Examiner—Cao (Kevin) Nguyen
_
Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
_
Anderson
6,317,141 B1 * 11/2001 Pavlay et al. ............. .. 345/732
*
(73) Assignee: FlashPoint Technology, Inc.,
Peterb0r0ugh> NH (Us)
(51)
(52)
(58)
*May 18, 2004
METHOD AND APPARATUS FOR CREATING
AN INTERACTIVE SLIDE SHOW IN A
C. Anderson, San Jose, CA (US)
(*)
US 6,738,075 B1
heterogeneous media objects includes at least one media
type such as, a still image, video, and audio. When the slide
’
shoW is subsequently presented, each one of the media
Int. Cl.7 .............................................. .. H04N 5/222
US. Cl. ................................. .. 345/723, 348/333.05
Field of Search ............................... .. 345/723, 732,
objects in the slide shoW is automatically displayed on a
display the Order seleeted- Aeeerdihg to the present
invention, the user is able to Change the Order the media
345/734, 788; 34853305, 333, 332, 352
objects Will be played back, and able to control hoW long
each one of the media objects Will be played back, to thereby
(56)
References Cited
interactively present the slide shoW from the digital imaging
U.S. PATENT DOCUMENTS
6,038,545 A *
devlcej
3/2000 Mandeberg et al. ........ .. 705/15
14 Claims, 13 Drawing Sheets
140
302 —\
\
a
"‘
{>
Image #314
12/4/96
Malrk
\308a
Edit
\308b
_ 306
10:26AM
Save
\308c
2206a 2206b 2206C
__
7"‘ 310
U.S. Patent
May 18,2004
Sheet 2 0f 13
A
Mark
View
x
US 6,738,075 B1
140
Edit
110
J
r
\
R 22060 2206b 22060
o
<2
202
FIG. 2A
204
200
\
100
210
212
208
FIG. 2B
/
U.S. Patent
May 18,2004
Icon
Sheet 3 0f 13
Media Types
Single Still Image
Burst or Time Lapse
Panorama
Video
Audio Clip
Text File
FIG. 3
US 6,738,075 B1
U.S. Patent
May 18,2004
3o4—\
Sheet 4 0f 13
f3“;
\
BE]
E-Mail
Addresse
s
300 —\ 80:0: K]
US 6,738,075 B1
[140
I5
/—302
E‘ _A(
H
[:1
H
K]
Ma_rk
Edit
\ 308a
Save
\ 308D
\ 3080
2206a 2206b 22066
FIG. 4A
302
140
350 —\
\
\
/
\
\
356 —“
E
— 352
’_‘
354j
D
[II [Ii]
Image #314
12/4/96
Malrk
Edit
10:26AM
Saye
2206a 2206b 2.2060
FIG. 4B
_ 306
__‘
f— 310
U.S. Patent
May 18,2004
US 6,738,075 B1
Sheet 5 0f 13
I 500
Select Image
[- 502
Press Mark ‘
Function Key
(-504
Select Nexi
Dispiey Mark
’ Image
Number
{506
Update Mark So?key
to Unmark
508
More
Images to
Mark 7
Remove
image from
Group?
/ 516
Select Image
to Remove
r 514
1
r 518
Select a Function
to Apply to the Group
Press Unmark
1-
r 520
Remove Mark
Number from cell
524 \
'
s22
Renumber a"
Higher Number
Highest
number
Marks
Mark‘?
FIG. 5
/. 526
Update Unmam So?key '
to Mark
U.S. Patent
May 18,2004
Sheet 6 0f 13
US 6,738,075 B1
/~140
300
E-Mml I
EU
Addresse E
1H
?ll
[IIIII
S
W
/—302
mm
U
B
H
|Kl
Malrk
Edit
Saye
\sosa
\soab \sosc
2206a 2206b 22060
FlG. 6
/—140
[E] AS02822;
3001 \
m
[[m]
0|
1 lmlfB
"<1
322
Malrk 1] I Edit
2
\ 308a
\ 308D
Sav_e
\ 3080
2206a 2206b 22060
FIG. 7
U.S. Patent
May 18,2004
Sheet 7 0f 13
US 6,738,075 B1
[140
342
\
|:|
E-Mail
3
K]
Addresse
Sound
"3
lie
El
2 E
Note
Mark
\ 308a
s
m
E]
Edit
\ 308b
E]
in
1
5
Save
\ 3080
2206a 220Gb 2206C
FIG. 8
f 360
Meta Data File
v/ 360'
Directory
Pointer to Media Object 6
Position(1)
Duration of Play(5 sec.)
Pointer to Audio File
E3 /Ftoot
_Ej I-d Sh
SI 9
OW
1
—% Media Object 6
Pointer to Media Object 8
P0sition(2)
——@ Media Object 8
Duration of Play(Auto)
P ' te to A d'o F'Ie
oin
r
u l
l
——@Media Objectt
Pointer to Media Object 1
Position(3)
Duration of Play(3 sec.)
Pointer to Audio File
FIG. 9A
FIG. 9B
U.S. Patent
May 18,2004
Sheet 8 0f 13
US 6,738,075 B1
380
100
302
350 -—\\
140
l/—
\
<>
/—
g
I
£
- 352
/~—§ |_—EditObject 406 l
Slide Show
Edit Screen
404
\
m
<>|
Properties 408
|
_<>|
"""" "
I
l
—
<>r
Choose
Help
-------- -
|
Exit
-— 402
U.S. Patent
May 18,2004
422 —\
424 —\
\
‘
Image
US 6,738,075 B1
Sheet 9 0f 13
Ori
m g
inai
\
Modified‘
140
a
/’——§
“Mg
—352
_
—
404
Screen
\
<> Bnhtness
w
_ 402
0 IEI
Choose |
Help
I ?Exit
\sosa Q\308b
2206a
i- 310
\sosc
206b
2060
FIG. 12
432
436
\
O
\
Q
:
f
\
O’
:
404 _
*
<>
434
“6°
Screen
/
m
- 352
—
\-. I Preview
|
<> @I:
Editing
438
— 402
0 ‘II
"
....... -.
440 __/_
OF
Zoom in
\soaa
2205a
_
Zoom Out |
l
Exit
__|— 310
\sosb
\sosc
206b
2060
FIG. 13
U.S. Patent
May 18,2004
Sheet 10 0f 13
4321
O
O
\
O’
:
<>
— 352
434
_
__R<> | Preview
Editing
Screen
438
/F
:
404
Video
US 6,738,075 B1
|
Q [El
m
/
_ 402
O E:
I
- - - - - - - -
440 —/
0|
Insert
1
Exit
1
\sosa
J— 310
\aosb Q\3os<>
2206a
206b
2060
FIG. 14
432
436
\
O
O
V'd
'
\
\
Editing
f
0/
:
<>
434
_\-
438
—
:
404
e0
442
/
— 352
—
—'
0 Prev|ew
Screen
0 IE‘
m
0 E:
‘ 402
440 —/
Choose
Move
\soaa
2206a
|
Delete
\soab Q\30sc
206b
FIG. 15
2060
|— 310
U.S. Patent
May 18,2004
Sheet 11 0f 13
US 6,738,075 B1
/_444%—438
Video
Editing
OIKI
(NI!
Screen
m
Copy
Move
\aosa
2206a
I
Delete
\sosb
\sosc
206b
2060
FIG. 16
436 —\
<)
-352
Video
Editing
(NC!
OIII
Screen
@
Insert
\sosa
2206a
Replace
|
Undo
\soab
\soac
206b
2060
FIG. 17
— 402
U.S. Patent
May 18,2004
Sheet 12 0f 13
US 6,738,075 B1
452 —\
[438
<> \ <>
0’
—
OMAWMANVVO
—- 352
' . 1“ 4 34
Audio
404—
\~§E:|
Editing
__
GEE:
m
3
Zoom In
E:
Zoom Out
\sosa
|
Exit
‘402
j— 310
\soab Q\30sc
2206a
206b
2060
FIG. 18
466——\
The Seventh Sy
hony shows
evidence of a
——
n style
- 352
464 x from his earlier ones
_
Change
Teglfedégng
m
(J b 0 cl e f g h i j
462
—4o2
klmnopqrst
\u v w x y z
Type
|
\sosa
2205a
Shift
,
|
.
/
__
Exit
|-— 310
\sosb
\sosc
206b
2060
FIG. 19
U.S. Patent
May 18, 2004
Sheet 13 0f 13
US 6,738,075 B1
Play List
2000
Previous
‘
200a
q @ >
Next
20%
V
200d
FIG. 20
482 —\
Properties
Screen
4Q
140
/4 FPosmo=ma| |3
|
0 [Duration:
]
| I3 Sec.
\—O | 1st Soft KeyzJ lObject #8
|
'— 402
<>|2nd Soft_Key: | mbject #20 j
J
Choose |
\sosa
2206a
—
Help
|
|— 310
Exit
\sosb
206b
FIG. 21
206C
US 6,738,075 B1
1
2
Since digital cameras capture images and sound in digital
METHOD AND APPARATUS FOR CREATING
AN INTERACTIVE SLIDE SHOW IN A
DIGITAL IMAGING DEVICE
format, their use for creation of multimedia presentations is
ideal. HoWever, despite their capability to record still
images, audio, and video, today’s digital cameras require the
CROSS-REFERENCE TO RELATED
APPLICATIONS
user to be very technologically pro?cient in order to create
multimedia presentations.
The present invention is related to the following
co-pending US. Patent Applications:: Ser. No. 09/223,962
entitled “Method And Apparatus For Creating A Multimedia
Presentation From Heterogeneous Media Objects In ADigi
tal Imaging Device,” and Ser. No. 09/223,960 entitled
“Method And Apparatus For Editing Heterogeneous Media
Objects In A Digital Imaging Device”, both ?led concur
rently hereWith; and Ser. No. 08/716,018 entitled “Method
And System For Displaying Images And Associated Media
Types In The Interface Of A Digital Camera,” ?led Sep. 9,
10
such as Microsoft PoWerPointTM. The user may also edit the
images and video using any one of a number of image
editing softWare applications. After the PoWerPoint presen
15
using a remote control.
Due to the limitations of today’s digital cameras in terms
FIELD OF THE INVENTION
20
ing device and more particularly to a method and apparatus
for creating, editing and presenting a multimedia presenta
tion comprising heterogeneous media objects in the digital
imaging device.
25
BACKGROUND OF THE INVENTION
becomes increasingly mainstream, hoWever, the number of
novice computer users Will increase. Indeed, many users Will
oWners Will be precluded from taking advantage of the
The use of digital cameras is rapidly proliferating and
multimedia capabilities provided by digital cameras.
30
What is needed is an improved method for creating,
editing, and displaying a multimedia presentation using
images and/or video from a digital. imaging device. The
present invention addresses such a need.
native to digital video cameras. Still digital cameras are 35
typically less expensive because they have far less process
ing poWer and memory capacity than digital video cameras.
SUMMARY OF THE INVENTION
The present invention provides a method and apparatus
for interactively presenting a slide shoW in a digital imaging
device. The method aspect of the present invention includes
?rst creating a slide shoW from randomly selected ones of
Digital video cameras differ from digital still cameras in
a number of respects. Digital video cameras are used to
capture video at approximately thirty frames per second at
the expense of image quality. Digital video cameras are
of capabilities and features, the user is forced to learn hoW
to operate a computer, image editing softWare, and a pre
sentation program in order to effectively create and display
the multimedia presentation. As the use of digital cameras
not even oWn a computer at all. Therefore, many camera
they may one day overtake 35 mm SLR’s in terms of
WorldWide sales. There are basically three types of digital
cameras; digital still cameras, digital video cameras, and
hybrid digital-video cameras.
Still digital cameras are used primarily for capturing high
quality static photographs, and offer a less expensive alter
tation has been created, the user must connect the PC or
notebook to a projector to display the presentation. Finally,
the user typically controls the play back of the presentation
1996.
The present invention relates generally to a digital imag
For example, in order to create a multimedia presentation,
the user ?rst captures desired images and video With the
camera, and then doWnloads the images to a personal
computer or notebook computer. There, the user may import
the images and video directly into a presentation program,
40
the heterogeneous media objects stored in the digital imag
ing device, Where each one of the heterogeneous media
more expensive than still cameras because of the extra
objects includes at least one media type such as, a still
hardWare needed. The uncompressed digital video signals
from all the loW-resolution images require huge amounts
memory storage, and high-ratio real-time compression
image, video, and audio. When the slide shoW is subse
quently presented, each one of the media objects in the slide
shoW is automatically displayed on a display the order
selected. According to the present invention, the user is able
to change the order the media objects Will be played back,
and able to control hoW long each one of the media objects
45
schemes, such as MPEG, are essential for providing digital
video for today’s computers. Until recently, most digital
video recorders used digital magnetic tape as the primary
storage media, Which has the disadvantage of not alloWing
Will be played back, to thereby interactively present the slide
shoW from the digital imaging device.
random access to the data.
Hybrid digital-video cameras, also referred to as multi
media recorders, are capable of capturing both still JPEG
images and video clips, With or Without sound. One such
camera, the M2 Multimedia Recorder by Hitachi America,
Ltd., Brisbane, Calif., stores the images on a PC card hard
disk (PCMCIA Type III), Which provides random access to
the recorded video data.
All three types of cameras typically include a liquid
crystal display (LCD) or other type of display screen on the
back of the camera. Through the use of the LCD, the digital
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram illustrating of one preferred
embodiment of a digital video camera (DVC) for use in
55
60
cameras operate in one of tWo modes, record and play. In
record mode, the display is used as a vieW?nder in Which the
video. The camera may also be connected to a television for
displaying the images on a larger screen.
FIGS. 2A and 2B are diagrams depicting an exemplary
form factor design. for the DVC.
FIG. 3 is a table listing example media types that may be
captured and stored by the DVC.
FIGS. 4A—4B is a diagram illustrating one preferred
embodiment of the revieW mode screen.
FIG. 5 is a ?oWchart depicting the process of creating an
user may vieW an object or scene before taking a picture. In
play mode, the display is used a playback screen for alloW
ing the user to revieW previously captured images and/or
accordance With the present invention;
65
ordered group of heterogeneous media objects in accordance
With the present invention.
FIGS. 6—8 are diagrams illustrating examples of marking
heterogeneous media objects.
US 6,738,075 B1
4
3
and still images, an audio subsystem 142 for playing and
recording audio, buttons and dials 146 for operating the
DVC 100, and an optional status display 148.
FIGS. 9A—9B is a diagram illustrating a slide show object
implemented as a metadata ?le.
FIG. 10 is a diagram illustrating the DVC connected to
external projector, and alternatively to a television.
FIG. 11 is a diagram illustrating the components of the
slide-shoW edit screen in accordance With the present inven
tion.
FIG. 12 is a diagram illustrating the image editing screen.
FIG. 13 is a diagram illustrating the video editing screen.
FIGS. 14—17 are diagrams illustrating the process of
editing a video on the DVC by creating and moving a video
The CPU 124 may include a conventional microprocessor
device for controlling the overall operation of camera. In the
preferred embodiment, The CPU 124 is capable of concur
rently running multiple softWare routines to control the
various processes of camera Within a multithreaded envi
ronment. In a preferred embodiment, The CPU 124 runs an
10
Although the CPU 124 is preferably a microprocessor, one
or more DSP 116’s (digital signal processor) or ASIC’s
clip.
FIG. 18 is a diagram illustrating an audio editing screen
for editing audio media types.
operating system that includes a menu-driven GUI. An
example of such softWare is the DigitaTM Operating Envi
ronment by FlashPoint Technology of San Jose, Calif.
15
(Application Speci?c Integrated Circuit) could also be used.
Non-volatile memory 128, Which may typically comprise
FIG. 19 is a diagram illustrating a text editing screen for
a conventional read-only memory or ?ash memory, stores a
editing text media types.
FIG. 20 is a diagram illustrating the mapping of the
executed by the CPU 124. Input/Output interface (I/O) 150
four-Way control during slide shoW presentation.
is an interface device alloWing communications to and from
set of computer readable program instructions that are
computer 112. For example, I/O 150 permits an external host
computer (not shoWn) to connect to and communicate With
computer 118.
FIG. 21 is a diagram illustrating the properties page of a
media object.
DETAILED DESCRIPTION OF THE
INVENTION
25
The present invention is a method and apparatus for
creating and presenting a multimedia presentation compris
ing heterogeneous media objects stored in a digital imaging
device. The folloWing description is presented to enable one
of ordinary skill in the art to make and use the invention and
is provided in the context of a patent application and its
requirements. Although the present invention Will be
described in the context of a digital video camera, various
modi?cations to the preferred embodiment Will be readily
apparent to those skilled in the art and the generic principles
herein may be applied to other embodiments. That is, any
digital imaging device used to store and display and/or
video, could incorporate the features described hereinbeloW
and that device Would be Within the spirit and scope of the
present invention. Thus, the present invention is not
35
embodiment, poWer supply 134 provides operating poWer to
a main poWer bus 152 and also to a secondary poWer bus
45
external poWer source. During normal operation of poWer
supply 134, the main batteries (not shoWn) provide operat
ing poWer to poWer supply 134 Which then provides the
operating poWer to camera via both main poWer bus 152 and
secondary poWer bus 154. During a poWer failure mode in
The DVC 100 comprises an imaging device 110, a com
puter 112, and a hardWare user interface 114. The Imaging
device 110 includes an image sensor (not shoWn), such as a
charged coupled device (CCD) or a CMOS sensor, for
Which the main batteries have failed (When their output
voltage has fallen beloW a minimum operational voltage
55
frames are transferred from the imaging device 110 to the
hardWare user interface 114.
The computer 112 includes an image processing digital
signal-processor (DSP) 116, a video codec 132, an audio
clam-shell design having a rotatable imaging device 110.
FIG. 2A is a top vieW of the DVC 100 in an opened position,
While FIG. 2B is a top vieW of the DVC 100 in a closed
position. FIG. 2A shoWs the display screen 140, a four-Way
navigation control 200, a mode dial 202, a display button
204, a set of programmable soft keys 206, a shutter button
codec 132, a mass storage device 122, a CPU 124, a DRAM
126, an internal nonvolatile memory, a mixer, and a video
control 132. The computer 112 also includes a poWer supply
134, a poWer manager 136, and a system bus 138 for
The hardWare interface 114 for interacting With the user
includes a display screen 140 for displaying the digital video
level) the backup batteries provide operating poWer to poWer
supply 134 Which then provides the operating poWer only to
the secondary poWer bus 154 of camera.
FIGS. 2A and 2B are diagrams depicting an exemplary
form factor design for the DVC 100, shoWn here as a
computer 112 for processing, storage, and display on the
connecting the main components of the computer 112.
154. The main poWer bus 152 provides poWer to imaging
device 110, I/O 150, Non-volatile memory 128 and remov
able memory. The secondary poWer bus 154 provides poWer
to poWer manager 136, CPU 124 and DRAM 126.
PoWer supply 134 is connected to main batteries and also
to backup batteries 360. In the preferred embodiment, a
camera user may also connect poWer supply 134 to an
is preferably capable of capturing and displaying various
types of image data including digital video and high
resolution still images.
capturing frames of image data in bayer format. The image
components of camera. PoWer manager 136 communicates
via line With poWer supply 134 and coordinates poWer
management operations for camera. In the preferred
intended to be limited to the embodiment shoWn but is to be
accorded the Widest scope consistent With the principles and
features described herein.
Referring noW to FIG. 1, a block diagram of one preferred
embodiment of a digital video camera (DVC) is shoWn for
use in accordance With the present invention. The DVC 100
Dynamic Random-Access-Memory (DRAM) 126 is a
contiguous block of dynamic memory that may be selec
tively allocated for various storage functions. DRAM 126
temporarily stores both raW and compressed image data and
is also used by CPU 124 While executing the softWare
routines used Within computer 112. The raW image data
received from imaging device 110 is temporarily stored in
several input buffers (not shoWn) Within DRAM 126. A
frame buffer (not shoWn) is used to store still image and
graphics data via the video control 132 and/or the mixer.
PoWer supply 134 supplies operating poWer to the various
65
208, a menu button 210, and an audio record button 212.
The mode dial 202 is used to select the operating modes
for DVC 100, Which include a capture mode (C) for record
US 6,738,075 B1
5
6
ing video clips and for capturing images, a review mode (R)
for quickly viewing the video clips and images on the
display screen 140, and a play mode (P) for vieWing
in order to save bus bandWidth. HoWever, if suf?cient bus
bandWidth is provided (e.g., 100 MHZ), the video stream
could be ?rst stored in memory.
Although the resolution of the display screen 140 may
vary, the display screen 140 resolution is usually much less
full-siZed images on the display screen 140.
When the DVC 100 is placed into capture mode and the
than the resolution of the image data that’s produced by
display screen 140 is activated, the camera displays a “live
vieW” of the scene vieWed through the camera lens on the
display screen 140 as a successive series of real-time frames.
If the display screen 140 is not activated, then the user may
vieW the scene through a conventional optical vieW?nder
imaging device 110 When the user captures a still image at
full resolution. Typically, the resolution of display screen
140 is 1A the video resolution of a full resolution image.
10
(not shoWn).
Referring to FIGS. 1 and 2A, during live vieW, the
imaging device 110 transfers raW image data to the image
processing DSP 116 at 30 frames per second (fps), or 60
?elds per second. The DSP 116 performs gamma correction
and color conversion, and extracts exposure, focus, and
White balance settings from the image data and converts the
data into CCIR 650 streaming video. (CCIR 650 is an
international standard for digital video designed to encom
pass both NTSC and PAL analog signals, providing an
NTSC-equivalent resolution of 720x486 pixels at 30 fps. It
vieW process are also 1A resolution.
As stated above, the DVC 100 is capable of capturing
high-resolution still images in addition to video. When the
15 user initiates the capture function to capture a still or
sequential image, the image device captures a frame of
image data at a resolution set by user. The DSP 116 performs
image processing on the raW CCD data to convert the frame
of data into YCC color format, typically YCC 2:2:2 format
(YCC is an abbreviation for Luminance, Chrominance-red
and Chrominance-blue). Alternatively, the data could be
converted into RGB format (Red, Green, Blue).
After the still image has been processed, the image is
requires 27 MB per second and uses three signals: one 13.5
MB/sec luminance (gray scale) and tWo 6.75 MB/sec
chrominance (color)).
After processing, the streaming video from the DSP 116
is transferred to the mixer for the overlay of optional
graphics and/or images onto the video. The graphics data
Since the display screen 140 is capable of only displaying
images at 1A resolution, the images generated during the live
25
compressed, typically in JPEG format, and stored as an
image ?le on the mass storage device 122. A JPEG engine
(not shoWn) for compressing and decompressing the still
images may be provided in the image processing DSP 116,
from the DRAM’s 126 frame buffer is transferred to the
the video codec 132, provided as a separate, unit, or per
mixer in synch With streaming video, Where the mixer
formed in softWare by the CPU 124.
combines the graphic data With the video. After the stream
ing video and the graphics are combined, the video is
After the image has been compressed and stored, live
vieW resumes to alloW the capture of another image. The
user may continue to either capture still images, capture
video, or sWitch to play or revieW mode to playback and
displayed on the display screen 140 via the video control
132. A video out port is also provided to display the video
on an external display device.
vieW the previously stored video and images on the display
35
When the user initiates the video capture function to
screen 140. In a preferred embodiment, the DVC 100 is
record the digital video, the streaming video output from the
capable of capturing several different media types, as shoWn
DSP 116 is also transferred to the video codec 132 for
in FIG. 3.
FIG. 3 is a table listing example media types that may be
captured and stored by the DVC 100. Also shoWn are the
corresponding icons that are used to indicate to the media
type. The media types include a single still image, a time
lapse or burst image, a panorama, a video segment, an audio
clip, and a text ?le.
compression and storage. The video codec 132 performs
MPEG-2 encoding on the streaming video during recording,
and performs MPEG-2 decoding during playback. The video
codec 132 may include local memory, such as 32 Mbits of
SDRAM 126 for example, for MPEG-2 motion estimation
betWeen frames. Such video codecs 132 are commercially
available from Sony Electronics (CXD1922Q0) and Mat
sushita Electronics Corp.
As the video codec 132 compresses the digital video, the
45
A still image is a high-quality, single image that may have
a resolution of 1536x1024 pixels, for example. Atime-lapse
image is a series of images automatically captured by the
compressed video stream is transferred to a temporary buffer
DVC 100 at prede?ned time intervals for a de?ned duration
in DRAM 126. Simultaneously, audio is recorded by the
audio subsystem 142 and transferred to the audio codec 132
for compression into a compressed audio format, such
MPEG Audio Layer 3 (MP3), Which is common internet
(e.g. capturing a picture every ?ve minutes for an hour). A
burst image is similar to a time-lapse, but instead of cap
format. In an alternative embodiment, the audio could be
(e.g., a couple seconds). A panorama image is an image
comprising several overlapping images of a larger scene that
have been stitched together. A burst image, a timelapse
image, and a panorama image are each objects that include
multiple still images, therefore, they may be referred to as a
compressed into AC-3 format, a Well-knoWn Dolby Digital
audio recording technology that provides six surround
turing images for de?ned period of time, the DVC 100
captures as many images as possible in a brief time frame
55
sound audio channels.
The CPU 124 mixes the compressed video and audio into
a speci?ed format, such as MPEG-2, for example. After the
compressed MPEG-2 data is generated, the CPU 124 trans
fers the MPEG-2 data to the removable mass-storage device
122 for storage. In a preferred embodiment, the mass storage
device 122 comprises a randomly accessible 3-inch record
sequential images.
In addition to capturing different image-based media
types, the DVC 100 can capture other media types, such as
audio clips and text. The user can record a voice message to
create a stand-alone audio clip, or the user may record a
voice message and have it attached to an image to annotate
able DVD drive from Toshiba/Panasonic, or a one-inch 340
MB MicroDriveTM from IBM, for example.
The video architecture inputs the video stream from the
DSP 116 directly into the mixer, rather than ?rst storing the
video in memory and then inputting the video to the mixer,
65
the image. Audio clips may also be doWnloaded from an
external source to add sound tracks to the captured objects.
Atext media type is created by entering letters through the
buttons on the user interface. The text along With graphics
US 6,738,075 B1
7
8
can be overlaid as Watermarks on the images or, the text can
be saved in a ?le to create a text-based media type.
displaying the media type icons associated With the active
media object 302, a large thumbnail 354 shoWing a larger
vieW of the active media object 302, and the command bar
310.
In a preferred embodiment, the ?lmstrip 352 displays four
thumbnail images 350 at a time, although other numbers are
also suitable. The user may navigate through the series of
displayed thumbnails 350 in the display screen 140 using the
In a preferred embodiment, one or more of the different
media types can be combined to form a single media object.
Since various combinations may be formed, such as single
image With sound, or burst image With text, etc, the DVC
100 can be described at storing heterogeneous media
objects, each comprising a particular combination of media
types, such as images, video, sound, and texvgraphics. Some
types of media objects are formed automatically by the DVC
four-Way navigation control 200 (FIG. 2A). When the user
10
100, such as a captured image or an annotated image, others
are formed manually by the user.
After media objects are created and stored, the user may
camera contents. As the user presses the buttons on the
vieW the media objects by sWitching the camera to play
mode or revieW mode. In play mode, the camera 100 alloWs
the user to vieW screen-siZed images in the display screen
holds doWn the left/right buttons on the four-Way control
200, the thumbnails 350 are scrolled-off the display screen
140 and replaced by neW thumbnails 350 representing other
stored media objects to provide for fast broWsing of the
four-Way control 200 and the thumbnails 350 scroll across
15
the display screen 140, the thumbnail 350 that is positioned
over a notch in the selection arroW line 356 is considered the
active media object 302. When there are more than four
media objects in the camera, the selection arroW line 356
displays arroWheads to indicate movement in that direction
140 in the orientation that the image Was captured. Play
mode also alloWs the user to hear recorded sound associated
With a displayed image, and to play back sequential groups
of images (time lapse, burst, and panorama images) and to
is possible With the left/right navigation buttons.
vieW movies from the video.
In revieW mode, the DVC 100 enables the user to rapidly
revieW the contents of the DVC. In addition, the media
302, the media type icons corresponding to that media object
objects may be edited, sorted, printed, and transferred to an
external source.
When a thumbnail 350 becomes the active media object
are automatically displayed in the icon/information area
306, along With the large thumbnail 354. Other information
25 can also be displayed, such as the name or number of the
media object, and the date and time the media object Was
Referring noW to FIG. 4A, a diagram illustrating one
preferred embodiment of the revieW mode screen is shoWn.
captured or created, for example.
In both the ?rst and second embodiments of the revieW
screen layout, displaying icons and text information in the
icon/information area 306 according to the present invention
provides the user With an automatic method identifying
common groups of media objects. This also reduces the need
Moving the mode dial 202 (FIG. 2) to access the revieW
mode enables the user to vieW all the media objects in the
camera along With the speci?c media types associated With
each of the objects.
The ?rst embodiment of the revieW mode screen displays
a series of object cells 300 that represent the media objects
for the user to sWitch to play mode to vieW the full-siZed
stored on the DVC 100, and a command bar 310. The 35
display screen 140 is shoWn here as displaying nine object
cells 300, although other numbers are also suitable.
The user may navigate through a series of displayed
In a ?rst aspect of the present invention, a method and
objects stored and displayed on the DVC 100. This is
accomplished by navigating through several displays shoW
ing the heterogeneous media objects, selecting and marking
45
Each object cell 300 includes an image area 304 and an
icon/information area 306. In the case of a still image, the
image area 304 of a object cell 300 displays a thumbnail of
the media object, Which in the case of an image-based media
the desired objects in the preferred order to create an ordered
list of objects, and then saving the ordered list of objects as
a slide shoW, thereby creating a neW type of media object.
After the slide shoW is created, the user may present the slide
shoW Wherein each media object comprising the slide shoW
is automatically played back to the user in sequence that it
Was selected. The slide shoW may be played back on the
display screen 140 and/or on an external television via the
video out port.
object is a small, loW-resolution version of the image. In the
case of sequential images and video-segments, the image
In a second aspect of the present invention, each media
object may be edited before or after incorporation into the
area 304 of a object cell 300 displays a representative
thumbnail or frame from the image sequence or video,
slideshoW, Where each media object is edited using different
respectively, typically the ?rst one.
media types editors designed to edit the media types asso
The icon/information area 306 displays one or more 55
graphical icons and/or text information indicating to the user
What media types have been associated With the media
ciated With that particular object.
In a third aspect of the present invention, the user may
specify parameters for slide shoW so that the objects in the
object displayed in the image area 304. The icon/information
area 306 may be placed in various positions relative to the
image area 304. HoWever, in a preferred embodiment, the
icon/information area 306 is displayed on the right-hand side
of each object cell 300, as shoWn.
Referring noW to FIG. 4B a diagram illustrating a second
preferred embodiment of the revieW mode screen is shoWn,
Where like components share like reference numerals. In the
second preferred embodiment, the revieW mode screen
includes a ?lmstrip 352, the icon/information area 306 for
objects for display.
apparatus is provided for creating and presenting a multi
media presentation from the heterogeneous group of media
object cells 300 in the display screen 140 using the four-Way
navigation control 200. The object cell 300 currently
selected by the four-Way navigation control 200 is indicated
by a highlighted area 302, Which in this embodiment is
shoWn as selection rectangle. Other shapes or indications
that a object cell 300 is the currently active object cell are
also suitable.
vieW of the object in order to recall the object’s subject
matter, Which eliminates the need for decompressing the
slide shoW are not displayed linearly, but are displayed in an
order that is dependent upon user de?ned events, thus
creating an interactive slide shoW.
Each aspect of the present invention Will noW be
explained in the sections beloW.
Slide ShoW Creation From Heterogeneous Media
65
Objects
In a preferred embodiment, a slide shoW is generated by
providing the DVC 100 With a marking and unmarking
US 6,738,075 B1
10
function Within the user interface 114 that simultaneously
depressed, the object cell 300 is updated to display the
provides for the selection and order of the heterogeneous
media objects in the slide shoW.
Referring again to FIGS. 4A and 4B, in a preferred
number of images that have been marked during the current
sequence in step 504. The object cell 300 may also be
updated to display an optional graphic, such as a dog-ear
embodiment, the marking and unmarking function is imple
corner or a check mark, for eXample. After the object cell
mented through the use of the soft keys 206a, 206b, and
206c displayed in the command bar 310, Which are
300 has been updated, the “Mark” soft key in the command
bar is updated to “Unmark” in step 506.
programmable, i.e., they may be assigned prede?ned func
Next, the user decides Whether to add more media objects
to the temporary set of marked media objects in step 508. If
tions. Hence, the name “soft” keys.
The function currently assigned to a respective soft key
206 is indicated by several soft key labels 308a, 308b, and
10
selects the neXt media object using the four-Way naviga
tional control 200, and the “Unmark” soft key in the
command bar is updated to “Mark” in step 510.
308c displayed in the command bar 310 on the display
screen 140. In an alternative embodiment, the display screen
140 may be a touch-screen Wherein each soft key 206 and
corresponding label are implemented as distinct touch
sensitive areas in the command bar 310.
After a soft key label 308 has been displayed, the user
may press the corresponding soft key 206 to have the
function indicated by its label 308 applied to the current
image. The functions assigned to the soft keys 206 may be
changed in response to several different factors. The soft 206
keys may change automatically either in response to user
actions, or based on predetermined conditions eXisting in the
camera, such as the current operating mode, the image type
of the media object, and so on. The soft keys 206 may also
be changed manually by the user by pressing the menu
If the user decides not to add more media objects to the
15
objects from the group in step 512. If the user decides not to
remove any of the marked media objects from the group,
then the user may select a function, such as “Save” or
“Delete” to apply to the group in step 514.
If the user decides to remove a marked media object from
the group, then the group is dynamically modi?ed as fol
loWs. The user ?rst selects the media object to be removed
25
required on the user interface 114, and reducing the need to
access hierarchical menus.
In the ?rst embodiment of the present invention, the soft
35
media objects.
In general, the mark function indicated by soft key label
308a enables a user to create a temporary group of media
objects. After a group of media objects is created, the user
may then perform functions on the group other than trans
To create an ordered group of images, the user navigates
45
camera Will noW be explained by Way of a speci?c eXample
With reference to FIGS. 4A, 4B, and 6—8.
Referring again to FIG. 4A, assume that the user Wishes
object 302. At this point, the soft keys displayed in the
becomes a marked image. After an image is marked, the
“Mark” soft key label 308a is updated to “Unmark”. The
command bar are prompts to the user that the user may
“Unmark” function alloWs the user remove an image from
the group, Which removes the mark number from the object
cell 300 of the highlighted image.
55
using the four-Way navigation control 200, and the program
mable function keys 206, as shoWn in FIG. 5.
FIG. 5 is a ?oWchart depicting the process of creating an
ordered group of heterogeneous media objects in accordance
With the present invention.
The process begins When a user selects a media object by
positioning the highlight area 302 over the object cell 300,
or otherWise selects the object cell 300, using the four-Way
navigational control 200 in step 500. The user then presses
the function key corresponding to the Mark soft key label
308a in step 502. After the “Mark” soft key 206a is
marked group in step 522. If the selected media object is not
the highest numbered media object in the marked group,
then the marked media objects having a higher number are
renumbered by subtracting one from the respective mark
number and displaying the result in their object cells 300 in
step 524. After the mark number is removed from the
unmarked media object and the other mark numbers renum
bered if required, the “Unmark” soft key in the command bar
is updated to “Mark” in step 526. The user may then
to create a slide shoW beginning With the selected media
highlighted image 302 and the highlighted image 302
According to the present invention, a user may randomly
create an ordered group of heterogeneous media objects
navigational control 200 in step 516. The user then presses
continue to modify the group by marking and/or unmarking
other media objects accordingly.
The process of grouping media objects in the digital
forming the temporary group into a permanent slide shoW,
such as deleting the group and copying, for eXample.
to a particular media object using the four Way control 200
and presses the “Mark” soft key 206a corresponding to the
mark function indicated by soft key label 308a. In response,
a mark number is displayed in the object cell 300 of the
by selecting the marked media object using the four-Way
the function key corresponding to the “Unmark” soft key in
step 518.
After the “Unmark” key is depressed, the object cells 300
for the remaining marked media objects may be renumbered.
This is accomplished by determining Whether the selected
media object is the highest numbered media object in the
the number of functions that may be performed by the
camera, While both minimiZing the number of buttons
increase the number of functions the user could apply to the
temporary group of marked media objects in step 508, then
the user decides Whether to remove any of the marked media
button 210. Providing programmable soft keys 206 increases
keys 206 are “Mark”, “Edit”, and “Save”. Although not
shoWn, other levels of soft key functions may be provided to
the user decides to add more media objects, then the user
65
perform the displayed functions, such as “Mark”, on the
highlighted media object. The mark function is then per
formed by the user pressing the Mark function key 206a.
Referring noW to FIG. 6 a diagram illustrating the result
of the user pressing the Mark function key is shoWn. The
selected media object cell 302 is updated With the number
“1”, Which indicates that the media object is the ?rst to be
marked. FIG. 7 is a diagram shoWing the user marking
another media object by selecting a second media object cell
322 and pressing the Mark function key. This causes the
media object cell 322 to be updated With the number “2”.
FIG. 8 is a diagram shoWing a third media object being
selected and marked, as described above, in Which case, the
icon area of the media object 342 is updated With the number
“3”.
Referring again to FIG. 5, While marking media objects,
the method for removing media objects in the group (steps
US 6,738,075 B1
11
12
512—524) also allows a user to dynamically reorder or
displaying the image for a prede?ned time on the display
screen 140 While playing any, associated audio. Sequential
re-sequence the media objects in the group. For example,
images are played by displaying each still comprising the
sequential image While playing any associated audio. Video
assume the user has marked ?ve media objects, labeled as
“1”, “2”, “3”, “4”, “5”, and Wants to make media object “3”
the last media object in the group. This can be accomplished
segments are played as a convention movie. A text-based
by unmarking media object “3”, Which results in media
objects “4”, and “5” being renumbered “3” and “4”, respec
object is played by displaying the text on the display screen
140. And a stand-alone audio clip is played by displaying a
tively. Thereafter, the user may mark the original media
blank screen or the name of the clip While the audio is played
object “3”, Which results in the media object being labeled
With the number “5”.
10
DVC 100 to an external projector or television via the video
out port, and playing the slide shoW 360, the camera can be
used as a presentation device in place of a notebook
computer, as shoWn in FIG. 10.
Referring again to FIG. 4, after the group has been created
With the chosen media objects in the desired sequence, the
user saves the ordered group to create a slide shoW media
object. In a preferred embodiment, the slide shoW media
object is created using “Save” function shoWn in the com
mand bar 310.
15
In one preferred embodiment, pressing the soft key 206C
assigned the “Save” function creates a metadata ?le, Which
is a ?le containing data that describes other data.
Referring to FIG. 9A, a diagram illustrating a slide shoW
object 360 implemented as an exemplary metadata ?le is
shoWn. The metadata ?le includes a series of ?elds that acts
a play list When the ?le is read by identifying one or more
of the folloWing attributes for each media object:
a) A pointer to, or the address of, the media object,
b) An identi?cation of each media object’s associated
media types; and
c) A duration of play.
Creating a metadata ?le that simply points to the real media
objects saves storage space since the original media objects
large screen 384 or on the screen of the television 382 from
presentation.
25
Editing Media Objects
Referring again to FIG. 8 in a second aspect of the present
invention, the DVC 100 is provided With an advanced
feature that alloWs the user to edit the media objects either
before or after incorporation into the slide shoW 360 using
specialiZed media type editors. In one preferred
embodiment, the user edits the slide shoW 360 by selecting
In a second preferred embodiment, pressing the soft key
206C assigned the “Save” function (FIGS. 4A and 4B)
the slide shoW object in either revieW or play mode, and then
35
of the marked media objects either into a ?le, a folder, or a
directory on the DVC’s mass storage device 122. A dialog
box or other type of prompt appears asking the user to name
the neW ?le, folder, or directory.
Referring to FIG. 9B, a diagram illustrating a slide shoW
object 360‘ implemented as a ?le directory is shoWn. A
directory named “slide shoW” is created for the slide shoW
360‘, Where the name of the directory may be input by the
user. After the directory is created, each marked media
object is then copied to the directory as shoWn. Since the
media objects are copied, the original media objects are left
in tact, and the neW slide shoW object 360‘ may be trans
FIG. 10 is a diagram illustrating the DVC 100 connected
to external projector 380, and alternatively to a large tele
vision 382. When the slide shoW 360 is played, the images,
video and audio are automatically displayed directly on the
the DVC 100. Thus, the present invention enables a novice
user to shoW multimedia presentations Without the need for
doWnloading images and/or video to a computer for incor
poration into presentation softWare to create a multimedia
do not have to be duplicated.
creates a permanent group of media objects by copying all
through the DVC’s 100 speakers.
According to the present invention, by connecting the
pressing the “Edit” soft key 206b. In response a slide shoW
edit screen appears displaying the thumbnail images of all
the media objects in the slide shoW.
Referring noW to FIG. 11, a diagram illustrating the
components of the slide shoW edit screen is shoWn in
accordance With the present invention. The slide shoW edit
screen is based on the revieW screen layout of FIG. 4B,
45
Where like components share like reference numerals. The
slide shoW edit screen 400 includes, the ?lmstrip 352, a list
page 402, and the command bar 310. The ?lmstrip 352
displays a scrollable series of thumbnails representing all the
media objects in the slide shoW. The list page 402 displays
a scrollable list of menu items that can be applied to the
After the slide shoW 360 has been created using any of the
selected media object. And the command bar 310 displays
several of soft key functions 308.
described embodiments, it is displayed as a neW media
object cell 300 on the display screen 140 along With an icon
move a target cursor to discrete cursor locations 404 Within
indicating that the media object is a slide shoW. Selecting the
neW slide shoW object cell 300 and pressing the display
button 204 or sWitching to play mode causes each of the
the screen 400, shoWn here as diamond shapes, using the
four-Way navigational control 200. The cursor is active at
any given time in either the ?lmstrip 352 or the list page 402.
media objects included in the “slide shoW” to be individually
55 The current target-cursor location is shoWn as a black
ferred to an external source.
In the implementation shoWn in FIG. 11, the user may
played back on the display screen 140 in the sequence that
diamond, and the element associated With the current cursor
they Were marked Without user intervention.
In the case of a slide shoW 360 created as metadata ?le,
and played in the order listed in the ?le. In the case of a slide
shoW 360‘ created as a standard ?le or directory, the slide
location is the target element. In a preferred embodiment, the
soft key labels 308 displayed in the command bar 310 are
only associated With the target element.
To edit the slide shoW, the user navigates to the media
object of interest in the ?lmstrip 352 and presses the
“Choose” function 308a to select the targeted media object.
shoW 360‘ is played by displaying each media object in the
In response, the target cursor location in the noW inactive
the slide shoW is played by executing the metadata ?le,
causing each media object listed to be fetched from memory
order and listed.
When the slide shoW is presented, each media object
therein is played by playing each of the media types com
prising the object. For example, a still image is played by
?lmstrip 352 changes to a White diamond to shoW that the
65
selection of the selected media object 302 is persistent. At
the same time, the black diamond cursor appears in the
active list page 402.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement