r/106 r208

r/106 r208
US006389218B2
(12)
(54)
United States Patent
(10) Patent N0.:
Gordon et al.
45 Date of Patent:
METHOD AND APPARATUS FOR
6,253,375 B1 *
SIMULTANEOUSLY PRODUCING
COMPRESSED PLAY AND TRICK PLAY
Ma 14 a 2002
6/2001 Gordon et al. .............. .. 725/88
FOREIGN PATENT DOCUMENTS
BITSTREAMS FROM A VIDEO FRAME
EP
SEQUENCE
WO
WO96/13123
WO
96/17306
(75) Inventors: Donald F. Gordon, Los Altos; Ted
Ludvig, Redwood City, both of CA
(US)
0 701 371 A1
3/1996
*
........ .. H04N/7/173
5/1996
HO4N/5/76
6/1996
......... .. GO6F/15/16
OTHER PUBLICATIONS
Buddhikot et al “Design of a Large Multimedia Storage
Server” Computer Networks and ISDN Systems 27, pp.
(73) Assignee: DIVA Systems Corporation, Redwood
503_507, 1994,
City, CA (US)
( * ) Notice:
US 6,389,218 B2
_
_
* cited by eXammer
Subject' to any disclaimer, the term of this
Primary Examiner_vincent E BOCCiO
Patent 15 extended or adlusted under 35
(74) Attorney, Agent,
U.S.C. 154(1)) by 0 days.
Sheridan, LLP
or Firm—Moser, Patterson &
(21) Appl. N0.: 09/201,529
(57)
(22)
Filed:
An “all-in-one” encoder that simultaneously produces an
MPEG-2 compliant fast forward, fast reverse and play
(52)
(58)
US. Cl. .............. ..
386/68; 386/111; 386/112
Field of Search .......................... .. 386/6—8, 33, 68,
video. The encoder subsamples the video sequence, extract
ing a plurality of frames from the video sequence and
386/81, 92, 98, 109, 110, 111, 112, 124;
buffering the subsampled frames. Simultaneous with the
Int.
NOV_ 30’ 1998
(:1-7
. ... ..
. . . . . . . . . . . . . . . . . . . . . . . . . ..
ABSTRACT
bitstreams from a Sequence of Video frames, e_g_, 60l_format
360/48; H04N 5/91, 5/917
subsampling and buffering, the source frames are encoded
within a real-time encoder, e.g., an MPEG-2 encoder. The
(56)
References Cited
buffered frames are recalled from the buffer and coupled to
a second real-time encoder. The second encoder forms both
the fast forward and fast reverse bitstreams using a time
multiplexing technique wherein a group of pictures (GOP)
US. PATENT DOCUMENTS
5,481,297 A *
1/1996 Cash et a1,
348/1412
5,581,778 A * 12/1996 Chin et al. ............... .. 712/16
for the fast forward stream can be formed, followed by the
8/1997 Nalfai et a1- ---- -
386/106
compression of the same GOP having the frames organized
9/1997 Bleldt (“ft ‘11- ~~~~~~~ ~~
345/723
in reverse order. As such, the compressed GOPs are repre
345/327
sented by the fast forward and fast reverse bitstreams. The
play’ fast forgafd’ and?flast regerse bgstreatgs for each GOP
5,655,052 A
*
5,671,377 A :
2 * 141‘;
§°gaml et a1‘
5,815,146 A
9/1998 Youden et a1. ..... ..
,
,
5,889,918 A
5,946,446 A *
6,026,213 A *
6,097,877 A *
ane
3/1999
8/1999
2/2000
8/2000
. . . . . . . . . . . . . . . .
. . . ..
Sakazaki et a1. ........ ..
Yanagihara ............. ..
Higurashi et a1.
Katayama et a1. .......... ..
-
386/68
are ,Orgamze ,mto 2,‘
e an Store on t 6 mass Storage
devlce (6%» dlsk dnve arr21y) 0f the ?le Servel
386/68
386/68
386/68
19 Claims, 3 Drawing Sheets
REAL -TIME
SOURCE
ENCODER
VIDEO
(PLAY STREAM)
TO
STORAGE
\l 02
FRAME
SUBSAMPLER
\104
r/106
m
r208
REAL-TIME
ENCODER
BUFFER
(TRICK PLAY STREAM)
ADDR
R
CONTROLLER
(J1 10
U.S. Patent
May 14, 2002
Sheet 3 of3
US 6,389,218 B2
F
oom
WE.
/(_/
...
8m\v
wowA‘
? :22
$8_
#
wow
$50to:69%E5mP2m8\v
5:
Sz_w92$EEm2Z?0/mE 8$20%>M
:ZEi28t1a“
$22.1m;
#
nwkz6m0o
:518If?N
E21 s8:531
US 6,389,218 B2
1
2
METHOD AND APPARATUS FOR
SIMULTANEOUSLY PRODUCING
COMPRESSED PLAY AND TRICK PLAY
BITSTREAMS FROM A VIDEO FRAME
simultaneously produces an MPEG-2 compliant fast
forWard, fast reverse and play bitstreams from a sequence of
video frames, e.g., 601-format video. The encoder of the
present invention subsamples the video sequence, extracting
SEQUENCE
a plurality of frames from the video sequence and buffering
the subsampled frames. Simultaneous With the subsampling
BACKGROUND OF THE DISCLOSURE
1. Field of the Invention
and buffering, the source frames are encoded Within a
The invention relates to video-on-demand systems and,
more particularly, the invention relates to a method and
apparatus for simultaneously generating compressed play
10
and trick play bitstreams from a video frame sequence.
2. Description of the Background Art
In a video-on-demand system such as the OnSetTM system
manufactured by DIVA Systems Corporation, a ?le server is
used for streaming video information to users (subscribers)
of the system. The OnSetTM system is described in US. Pat.
No. 6,253,375, issued Jun. 26, 2001, and the ?le server is
described in US. Pat. Nos. 5,671,377 and 5,581,778. The
disclosures of this application and these patents are incor
porated herein by reference. The OnSetTM system contains
service provider equipment coupled through an information
distribution netWork to subscriber equipment. This system
15
(e.g., disk drive array) of the ?le server.
20
BRIEF DESCRIPTION OF THE DRAWINGS
The teachings of the present invention can be readily
understood by considering the folloWing detailed descrip
tion in conjunction With the accompanying draWings, in
25
subscriber equipment and the service provider equipment
30
To facilitate understanding, identical reference numerals
have been used, Where possible, to designate identical
forWard stream and a fast reverse play (reWind) stream. The
compressed video, Where the fast forWard stream is a
35 elements that are common to the ?gures.
compressed signal containing only every Nth frame of the
DETAILED DESCRIPTION
uncompressed play video and the fast reverse stream is a
compressed signal containing every Nth frame of the
uncompressed play video played backWards. To store the
video information, each stream is divided into portions (e.g.,
N-byte segments of compressed data) and striped onto a disk
FIG. 1 depicts a block diagram of an all-in-one encoder
100 comprising a ?rst encoder 102, frame subsampler 104,
40
array. The disk array stores all the information that the ?le
server can directly access.
The process for generating the fast forWard and fast
reverse streams is time consuming and expensive. The
process requires a frame by frame analysis of the video
45
50
and fast reverse streams in real time such that a real time
program can be encoded and stored for almost immediate
use of VCR-like functions.
Therefore, there is a need in the art for an improved
encoding system for a video-on-demand system to ensure
real-time availability of fast forWard and fast reverse func
tions.
SUMMARY OF THE INVENTION
The disadvantages associated With the prior art are over
come by the invention of an “all-in-one” encoder that
The second encoder 108 operating in conjunction With the
subsampler 104 and the buffer 106 encodes a subsampled
trick play bitstreams or trick play streams). The ?rst encoder
(the play stream encoder 102) contains a real-time MPEG-2
encoder that produces an MPEG-2 compliant, compressed
video bitstream (a play stream) from a sequence of 601
format video frames. The second encoder (the trick play
stream encoder 108) is also an MPEG-2 real-time encoder.
encoded (compressed). For a fast reverse stream, a sequence
of frames in reverse order is created and encoded. Such
encoding can cost $17 to $20 per minute of video. This
encoding process can not be used for producing fast forWard
a frame buffer 106, a second encoder 108 and a controller
110. The ?rst encoder 102 encodes a source video sequence
in a conventional manner, i.e., compressing the source video
sequence in real-time as the frames are input to the encoder.
version of the source video sequence to form a fast forWard
and fast reverse bitstreams (collectively referred to herein as
stream to enable the frames to be encoded in an MPEG-2
compliant form, yet provide a fast forWard and fast reverse
effect When decoded. As such, the video sequence is pro
cessed to extract every Rth frame (i.e., one of every R
frames, Where R is an integer greater than 1) to form a fast
forWard sequence and then the fast forWard sequence is
FIG. 2 illustrates the method of the present invention used
to produce a fast forWard bitstream; and
FIG. 3 illustrates the method of the present invention used
to produce a fast reverse bitstream.
equipment and stores, for a given movie, a standard play
stream (i.e., an MPEG-2 compressed video bitstream), a fast
fast forWard and fast reverse streams are also MPEG-2
Which:
FIG. 1 depicts a block diagram of the all-in-one encoder
of the present invention;
executes the commands to ful?ll the purpose of the
command, e.g., play, fast forWard, reWind, stop or pause the
movie.
A ?le server forms a portion of the service provider
ing technique Wherein a group of pictures (GOP) for the fast
forWard stream can be formed, folloWed by the compression
of the same GOP having the frames organiZed in reverse
order. As such, the compressed GOPs are represented by the
fast forWard and fast reverse bitstreams. The play, fast
forWard, and fast reverse bitstreams for each GOP are
organiZe into a ?le and stored on the mass storage device
provides subscribers VCR-like controls to enable a sub
scriber to select information content, for example, a movie,
then play, fast forWard, reWind, pause, or stop the selected
movie. The subscriber enters control commands through the
real-time encoder, e.g., an MPEG-2 encoder. The buffered
frames are recalled from the buffer and coupled to a second
real-time encoder. The second encoder forms both the fast
forWard and fast reverse bitstreams using a time multiplex
For the folloWing discussion, frame numbering is used to
55
describe the temporal order in Which frames occur in source
material Where 1 is the ?rst frame in the source material
frame sequence and 2 is the second frame and so on. For the
folloWing description, R is an integer and de?nes a play
back speed multiplier Which has signi?cance in the trick
60
play processes. The speed multiplier R is a variable that is
established by the controller 110. The controller 110 is a
microprocessor or microcontroller as Well as support circuits
65
such as memory, clocks, poWer supplies, etc. that is pro
grammed to perform the control functions of the all in one
encoder 100.
Throughout this disclosure, the exemplary trick play
streams are fast forWard and fast reverse. Of course other
US 6,389,218 B2
3
4
forms of trick play streams may be generated using the
that have been buffered after subsampling at one of R rate
all-in-one encoder 100 of the present invention. If the
(e.g., R=2 in FIG. 3). At step 302, these frames are selected
multiplier R equals 2, playback (decoding) of a fast forWard
stream is tWice normal single speed. If R equals 3, playback
from the buffer in reverse order as illustrated at 304. The
reverse order frames are encoded, at step 306, to produce a
sequence of GOPs 308 that are stored in reverse time order
in memory as illustrated in sequence 310.
is three times normal play speed and so on. Generally, the
trick play streams include a nine times fast forWard stream
and a nine times fast reverse stream. Alternatively, 32 times
normal fast forWard and fast reverse streams are also avail
able. The term source or source video is used to describe the
uncompressed video material such as 601-format video from
Which the trick play bitstreams are generated.
Using the all-in-one encoder 100, a fast forWard MPEG
video bitstream is generated from an uncompressed video
source such that When the stream is played back linearly
10
these teachings.
What is claimed is:
1. Apparatus for producing a plurality of encoded video
bitstreams from a video sequence comprising:
a ?rst encoder for producing a ?rst bitstream that contains
through a standard MPEG compliant video decoder, the
resulting imagery contains every Rth frame of the original
video sequence. As such, the display of the sequence has a
information that, When decoded, produces a standard
forWard play video frame sequence;
fast forWard effect.
A fast reverse stream is simultaneously produced by the
a frame subsampler;
a buffer that stores subsampled frames of the video
sequence;
a second encoder for producing a second bitstream that
all-in-one encoder such that When linearly played back
through a an MPEG compliant video decoder, the decoded
stream produces a sequence of frames that play in reverse
relative to the original frame sequence.
The play bitstream is formed using a conventional
MPEG-compliant encoder 102 that compresses a plurality of
Although various embodiments Which incorporate the
teachings of the present invention have been shoWn and
described in detail herein, those skilled in the art can readily
devise many other varied embodiments that still incorporate
contains information that, When decoded, produces a
?rst trick play frame sequence;
25
a third encoder for producing a third bitstream that
contains information that, When decoded, produces a
second trick play video frame sequence; and
frames (N frames that form a group-of-pictures (GOP)). As
is knoWn in the art, an N-frame GOP comprises at least one
a controller that selects subsampled frames from the
buffer and couples the selected frames to the second
and third encoders, Wherein at least one of said second
and third bitstreams comprises both intra-coded and
(e.g., one) intra-coded frame (i.e., an I-frame) and at least
one (e.g., N-1) inter-coded frames (i.e., forWard predicted
P-frames and/or bi-directionally predicted B-frames). To
form the trick play streams having N frame GOPs, the
inter-coded video frames, including inter-coded video
subsampler 104 extracts one out of every R consecutive
frames having non-Zero motion vectors.
2. The apparatus of claim 1 Wherein said ?rst encoder is
frames (arranged in increasing time code order) from the
source sequence. The selected N frames are buffered in the 35
an MPEG encoder that encodes N frames of the video
sequence.
frame buffer 106. For example, if R is tWo and the buffer
stores N frames, the buffer stores frames 1, 3, 5, 7, 9, and so
3. The apparatus of claim 1 Wherein said second and third
encoders are MPEG encoders that encodes R/N frames of
on up to frame 2N+1. The buffer 106 stores N successive
frames that de?ne a GOP for the trick play bitstreams. As
the video sequence having N frames.
4. The apparatus of claim 1 Wherein said ?rst trick play
such, Where multiple frames (i.e., every Rth frame) are
selected, the selected frames include frames to be intra
coded and frames to be inter-coded. Since the selected
frames to be interceded often represent a moving image,
such inter-coded frames have associate With them non-Zero
motion vectors.
video frame sequence is a fast forWard video frame
sequence.
5. The apparatus of claim 1 Wherein said second trick play
video frame sequence is a fast reverse video frame sequence.
45
FIG. 2 illustrates the process by Which a fast forWard
stream is produced. Sequence 200 represents the RN frames
(frames 1 to RN+1) that have been subsampled from a video
frame sequence at a one of R rate from a video frame
sequence and stored in the buffer (e.g., R=2, then 2N frames
are subsampled to form a trick play GOP). These buffered
frames are recalled from memory and encoded and then
associated With a normal play stream having a GOP of N
frames. The encoder (at step 202) repetitively produces a
sequence of GOPs 204 that are stored in increasing
addresses in memory as illustrated in sequence 206. The ?rst
55
6. The apparatus of claim 1 Wherein the controller mul
tipleXes selection of the frames from the buffer to apply a
plurality of subsampled frames to said second encoder to
form said second bitstream and then apply a plurality of
subsampled frames to said third encoder to form said third
bitstream.
7. A method of encoding frames of a video sequence to
form a plurality of bitstreams comprising the steps of:
encoding said frames to form a ?rst bitstream;
subsampling said frames;
buffering said subsampled frames;
GOP (GOP 0) contains frames 1 to RN+1, the second GOP
recalling said buffered frames in a forWard time sequence
(GOP 1) contains frames R(N+1)+1 through 2RN+1, the
third GOP (GOP 2) contains frames 2R(N+1)+1 through
encoding said recalled buffered frames to form a second
order;
bitstream;
3RN+1, and so on, Wherein each GOP is formed by at least
one I-frame, and one or more P-frames and/or B-frames 208.
recalling said buffered frames in a reverse time sequence
To generaliZe, a GOP contains frames GR(N+1)+1 through
(G+1)RN+1, Where G is the GOP number (e.g., 0, 1, 2,
order; and
encoding said recalled buffered frames to form a third
bitstream, Wherein at least one of said second and third
3, . . . ), R is the subsampling rate, and N is the number of
frames in a standard play GOP.
FIG. 3 illustrates the process by Which a fast reverse
stream is produced. Sequence 300 represents the RN frames
65
bitstreams comprises both intra-coded and inter-coded
video frames, including inter-coded video frames hav
ing non-Zero motion vectors.
US 6,389,218 B2
6
5
buffering said subsampled set of frames;
8. The method of claim 7 wherein said ?rst bitstream
When decoded forms a standard play frame sequence.
9. The method of claim 7 Wherein said second bistream,
When decoded, forms a fast forWard frame sequence.
10. The method of claim 7 Wherein said third bitstream,
recalling said buffered frames in a reverse time sequence
order; and
encoding said recalled buffered frames to form said
second bitstream.
15. The method of claim 14 Wherein said second
bitstream, When decoded, forms a fast reverse video frame
When decoded, forms a fast reverse frame sequence.
11. A method of encoding frames of a video sequence to
form a plurality of bitstreams comprising the steps of:
encoding said frames to form a ?rst bitstream;
While encoding said frames to form said ?rst bitstream,
selecting a subsampled set of frames from said frames
sequence.
16. The method of claim 11 further comprising the steps
10
buffering said subsampled frames of said frames in said
of said video sequence; and
encoding the subsampled set of frames to form a second
bitstream comprising both intra-coded and inter-coded
video frames, including inter-coded video frames hav
ing non-Zero motion vectors.
12. The method of claim 11 further comprising the steps
of:
buffering said subsampled set of frames;
of:
video sequence;
recalling said buffered frames in a forWard time sequence
15
order;
encoding said recalled buffered frames to form a second
bitstream;
recalling said buffered frames in a reverse time sequence
order;
encoding said recalled buffered frames to form a third
recalling said buffered frames in a forWard time sequence
bitstream.
order; and
17. The method of claim 16 Wherein said ?rst bitstream
encoding said recalled buffered frames to form said
When decoded forms a standard play frame sequence.
second bitstream.
18. The method of claim 16 Wherein said second bistream,
13. The method of claim 12 Wherein said second 25 When decoded, forms a fast forWard frame sequence.
19. The method of claim 16 Wherein said third bitstream,
bitstream, When decoded, forms a fast forWard video frame
When decoded, forms a fast reverse frame sequence.
sequence.
14. The method of claim 11 further comprising the steps
of:
*
*
*
*
*
UNITED STATES PATENT AND TRADEMARK OFFICE
CERTIFICATE OF CORRECTION
PATENT NO.
: 6,389,218 B2
DATED
I May 14, 2002
INVENTOR(S) : Gordon et at.
Page 1 0f 1
It is certified that error appears in the above-identi?ed patent and that said Letters Patent is
hereby corrected as shown below:
Column 3
Line 42, delete “interceded” and insert instead -- inter-coded
Signed and Sealed this
Fifteenth Day of April, 2003
JAMES E. ROGAN
Director ofthe United States Patent and Trademark O?‘ice
UNITED STATES PATENT AND TRADEMARK OFFICE
CERTIFICATE OF CORRECTION
PATENT NO. : 6,389,218 B2
DATED
: May 14, 2002
INVENTOR(S) : Gordon et at.
Page 1 of 1
It is certified that error appears in the above-identi?ed patent and that said Letters Patent is
hereby corrected as shown below:
Column 3
Line 42, delete “interceded” and insert instead -- inter-coded
Signed and Sealed this
Eighteenth Day of November, 2003
JAMES E. ROGAN
Director ofthe United States Patent and Trademark O?‘ice
UNITED STATES PATENT AND TRADEMARK OFFICE
CERTIFICATE OF CORRECTION
PATENT NO. : 6,389,218 B2
DATED
: May 14, 2002
INVENTOR(S) : Gordon et at.
Page 1 of 1
It is certified that error appears in the above-identified patent and that said Letters Patent is
hereby corrected as shown below:
Column 3
Between
Line 42, delete
lines 43
“interceded”
and 44, insert:
and insert instead -- inter-coded
~To produce a fast forward stream, the N frames from the buffer 106 are recalled under the
control of the controller 110 and coupled sequentially into the real-time encoder 108. The real-time
encoders 102 and 108 used herein may be any form of encoder that compresses video information.
One illustrative MPEG encoder is the E4 chip set available from C-Cubed of Spring?eld, Virginia. The
output is a compressed bitstream representing a fast forward GOP.
To produce a fast reverse stream, the N frames from the buffer 106 are recalled under the
control of the controller 110 and are coupled in reverse time order into the encoder 108. Although two
encoders could be used to produce each of the trick play streams independently, as tong as R equals
two or more, a single encoder can be used to produce both trick play tracks using time multiplexing
technique. To perform the multiplexed encoding, the fast forward GOP is encoded ?rst, then the fast
reverse GOP. After the buffer is ?lled with another N frames, the GOP pair is again encoded and so
on. The recall order of the frames from the buffer and the multiplexing process are controlled by
controller 110 by addressing the buffer in forward order for the fast forward GOP and in reverse order
for the fast reverse GOP.
The forward and reverse GOPs contain sequence start and sequence end codes such that,
when stored in a mass storage device, the bitstreams are clearly delimited.
The forward and reverse GOP bitstreams are stored with the play stream in a storage
medium such as a disk drive array or magneto-optical disk. The fast forward stream is written
sequentially, one GOP bitstream at a time, from the lowest address to the highest address. The
reverse bitstream is stored in reverse order from highest address to lowest address. The last byte in
the reverse GOP is placed in the highest address space of the reverse target ?le. As such, the
reverse GOP precedes, in address space, the ?rst frame of the immediately previously written reverse
GOP.
The entire source material frame sequence is encoded in this manner to simultaneously
produce a play, a fast forward, and fast reverse streams in real'timer
Signed and Sealed this
Second Day of December, 2003
JAMES E. ROGAN
Director ofthe United States Patent and Trademark O?‘ice
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement