W (144
US 20030038807A1
(19) United States
(12) Patent Application Publication (10) Pub. No.: US 2003/0038807 A1
(43) Pub. Date:
Demos et al.
(54)
METHOD AND APPARATUS FOR
PROVIDING COMPUTER-COMPATIBLE
(52)
FULLY SYNCHRONIZED AUDIO/VIDEO
INFORMATION
Feb. 27, 2003
US. Cl. ............................................................ .. 345/473
(57)
ABSTRACT
(76) Inventors: Gary Alfred Demos, Culver City, CA
(US); Peter Spoer, Burbank, CA (US)
An inventive display and input method and apparatus pro
viding fully synchronized audio/video information for use
With computers and for display on computer-compatible
monitors is described. The inventive display and input
system includes a computer interface, and provides synchro
Correspondence Address:
Martin J. J aquez, Esq.
.IAQUEZ & ASSOCIATES
Suite 2640
750 B Street
nized digital video and audio input, as Well as synchronized
digital and analog audio/video output. The display and input
system provides synchronization of moving images for
San Diego, CA 92101 (US)
display on 72 Hz and 75 Hz computer-compatible monitors.
(21) Appl. No.:
10/226,696
(22) Filed:
Aug. 22, 2002
The inventive display and input system facilitates 72 Hz and
75 Hz display of synchronized moving images, such as 24
fps tripled on 72 Hz computer-monitor displays, Without
using frame dropping or tearing techniques. In accordance
Related US. Application Data
(60)
With one aspect of the present invention, a “triple-repeat”
Provisional application No. 60/314,349, ?led on Aug.
22, 2001.
method (of 24 fps and 25 fps video) is used to provide
synchronized display onto 72 Hz and 75 Hz displays,
respectively. A “double-repeat” method is used to provide
synchronized display of 36 fps and 37.5 fps images on 72 Hz
and 75 Hz displays, respectively.
Publication Classi?cation
(51)
Int. Cl.7 ................................................... .. G06T 13/00
iii
128
Computer Interface
/
——
—
—
—
Optional Audio
/140
SyChronizationI
BufferAvailable142FT
|
Data
138
I
=
Buffer Memory
122—
t
|
I
4__ I,_116
Audio
_.>
I
I
I
I
Output ‘120 I
‘4;- ii 144
Color Space Transforms
124“~~
118 I
Audio
Input
‘
ii
I
d/
83%;" or
1747-60“ Synchronization 126
(optionally including U and V
System
resolution transformations)
-’
/
W (144
I
Digital
A to D
Video
Converter
104/I interface
‘\106
I
130
I
I
I
Digital Video
input
132
Analog Video
Input
RAMDAC I
I
I
I
Digital
I
Video
I Interface
110/’
72/75 Hz I
134 (I
Analog Vide
Output
\
11
4
12175 Hz
I
I
I
136
Digital Video
Output
I\
112
I
Optional Input Section
To. _
—
_
—
Optional Analog
Output Section
—
w;
100
Optional Digital
_
Video Output Section I
Patent Application Publication
a
Feb. 27, 2003 Sheet 1 0f 2
US 2003/0038807 A1
_I I NFOIH
[email protected]=c:2o5z30m5
JF
Feb. 27, 2003
US 2003/0038807 A1
METHOD AND APPARATUS FOR PROVIDING
COMPUTER-COMPATIBLE FULLY
SYNCHRONIZED AUDIO/VIDEO INFORMATION
units Were delivered to other in?uential facilities and people,
including employees of Jet Propulsion Laboratories (JPL),
and employees of AmpeX.
[0009]
CROSS-REFERENCE TO RELATED
PROVISIONAL APPLICATION
[0001]
This application claims the bene?t under 35 U.S.C.
§119(e) of pending U.S. Provisional Application No.
60/314,349, ?led Aug. 22, 2001, entitled “Synchronized
Display and Input System for Computers”, hereby incorpo
rated by reference herein in its entirety.
Gary Demos became one of a feW central ?gures in
the development of the ?eld of computer graphics. The early
frame buffer systems became important tools in the devel
opment of this ?eld, and Were highly in?uential With respect
to subsequent display and input system designs. The images
made With the early frame buffer designs Were seen around
the World in hundreds of presentations made betWeen 1975
and 1980.
[0010] In 1978, Gary Demos Worked With employees of
BACKGROUND OF THE INVENTION
[0002]
1. Field of the Invention
[0003] This invention relates to the ?eld of audio/video
signal processing, and more particularly to a method and
Information International to develop the ?rst high resolution
1000-line frame buffer computer display system. This frame
buffer doubled the resolution of the original “frame buffer”
system, extending the video image resolution to Well above
that of video systems, into the realm of “high de?nition”.
audio/video information for use by computers.
This display Was one of the very ?rst high de?nition system
components created, and it Was interfaced to a computer.
The display system may be considered as the very ?rst
[0004] 2. Description of Related Art
[0005] In 1973, Gary Demos (a co-inventor of the present
embodiment of a high de?nition computer display system.
[0011] In 1980, While in collaboration With digital video
invention), Working together With others, designed the ?rst
design experts employed by Information International, Gary
modern computer display and input system. This system Was
Demos designed the ?rst digital high-speed cross-color
apparatus for inputting and displaying fully synchroniZed
called a “frame buffer”, and it comprised a number of digital
correction system as part of a digital ?lm printer. The digital
and analog input and output ports, one of the digital in-and
?lm printer Was capable of performing generaliZed color
transformations, including the matriX transform subset. This
out ports being a computer (in this case a PDP-11 from
Digital Equipment Corporation). This system Was the very
system utiliZed 12-bit color precision on each of the three
?rst use of dynamic random access memories (DRAM’s),
even prior to their use in computers (Which at that time used
(usually red, green, and blue) color components. These
magnetic “Core” memories). This frame buffer also incor
porated a digital video lookup table to adjust colors prior to
systems became the ?rst embodiments of the computer
display and input system technology Which is noW ubiqui
tous in all personal and professional computer systems.
digital-to-analog (DAC) conversion for RGB display. This
[0012]
concept Would eventually be embodied in chips knoWn as
gration, of disparate and previously incompatible multi
“RAMDAC’s” a decade later.
media information types has taken place. For eXample, in
[0006] This ?rst “frame buffer” system had multiple
memory ports for the DRAM containing the image. This
“frame buffer” had a digital video input, digital and analog
video outputs, and the computer input and output ports. A
computer interface signal from the control interface of the
“frame buffer” indicated each frame display’s completion,
and could therefore be used to synchroniZe the image update
With the display refresh, for small WindoWs in the display
(since the computer at that time Was not fast enough to
update the entire screen in real time).
During the past feW years a convergence, or inte
recent years, integration of video information for use With
computers and computing peripherals has received consid
erable attention. Due to the massive memory and processing
requirements required by video, video and still image com
pression techniques have been developed to facilitate the
storage and processing of this information by computers.
HoWever, as is Well knoWn, because of inherent incompat
ibilities in the signal formats of computer graphics and
source video information (such as, for eXample, ?lm or
television video information), the integration of video infor
mation for use With computers has proven dif?cult and
[0007]
Another feature of this “frame buffer” Was its
ability to Zoom the image by integral piXel replication, and
then pan on the image via adjusting the starting scan address.
This feature could also be used to play short (loWer resolu
tion) movies Which ?t Within memory, by updating the
starting scan address to each frame in a sequence of frames
upon detection of the scan completion signal.
[0008] The initial system Was limited to 8-bits per piXel,
alloWing only a total of 256 colors. In 1974, further research
and design Was performed on the original Frame Buffer
concept. Gary Demos’ original design Was modi?ed by
others, and a ?rst modi?ed frame buffer system Was deliv
ered to the University of Utah. This modi?ed design Was
modi?ed further to support three sections of 8-bits each,
eXpensive.
[0013] Digital video processing equipment, such as digital
television receivers (DTVs), must be capable of inputting
and displaying myriad types of source video information
having different spatial and temporal resolution character
istics, and using different scanning formats. As is Well
knoWn, most analog video sources use an interlaced video
display format Wherein each frame is scanned out as tWo
?elds that are separated temporally and offset spatially in the
vertical direction. Although some variations eXist, the Well
knoWn NTSC video format (used throughout the Us. and
Japan) has a ?eld refresh rate of 60 HZ (actually 59.94 HZ)
interlaced. In Europe and elseWhere, PAL and SECAM color
composite video signals have a ?eld refresh rate of 50 HZ
alloWing the ?rst “frame buffer” implementation of 24-bit
interlaced. Motion pictures are predominantly produced
RGB color. This system Was delivered in 1975 to members
using a 24 frame per second rate. HoWever, in countries
using the PAL standard, motion pictures use a 25 fps rate.
of the NeW York Institute of Technology. Anumber of other
Feb. 27, 2003
US 2003/0038807 A1
[0014] In addition, several High De?nition Television
(HDTV) formats have been developed for the display of
digital imaging systems, proposals have been made in efforts
to standardiZe digital video processor designs and digital
high resolution television information. HDTV standards
have been proposed using 60 HZ interlaced ?elds, While
others have been proposed using 60 HZ progressively
video interfaces.
scanned images (i.e., non-interlaced). Varying aspect ratios
ommendations regarding the design of digital imaging sys
and horiZontal/vertical display resolutions have also been
proposed. For example, as described in the Society of
Motion Picture and Television Engineers (SMPTE) Stan
dard, SMPTE 292M, entitled “Bit-Serial Digital Interface
for High-De?nition Television Systems”, published in 1995
by the SMPTE and incorporated in its entirety herein by
[0018] For example, in the early 1990s, after receiving
input from industry experts, the SMPTE promulgated rec
tems. More speci?cally, the SMPTE generated a report of a
task force on “Digital Image Architecture” describing archi
tectural principles for designing digital moving image sys
tems. Gary Demos Was a co-editor and part author of this
report. Among the highlights of this report Was a recom
de?nition digital interface and its variants utiliZe a 74.25
MHZ pixel clock to carry a YUV (also referred to herein as
mendation that 72 HZ displays be considered for the pre
sentation of 24 fps ?lm material. The SMPTE report noted
that 72 HZ display refresh rates produce much less ?icker
than do 60 HZ display refresh rates used in NTSC television
systems, and is therefore more suitable for computer-com
YCrCb) (half horiZontal U and V resolution) digital 10-bit
patible display of moving images. The SMPTE report also
video signal for several high de?nition formats. These
formats include 1920 (horiZontal)><540 (vertical) resolution
interlaced ?elds at 60 ?elds per second, 1920 (horiZontal)><
1080 (vertical) at 24 frames per second (“fps”), and 1280
noted that 75 HZ display refresh rates be considered for PAL
reference, several high de?nition formats are utiliZed by
modern digital video equipment. The SMPTE 292M high
(horiZontal)><720 (vertical) at 60 fps.
[0015] Adding to the dif?culty of providing video infor
mation for use With computers is the fact that computer
(50 HZ) countries, Which shoW ?lm at 25 fps. Although most
computer display systems Which used Cathode Ray Tube
(CRT) displays had increased their display refresh rates to
exceed 60 HZ to reduce ?icker, the SMPTE task force report
recommendation of increasing the display rates represented
a departure from previous television system display rates.
monitors typically provide a much higher resolution than do
[0019] Although the SMPTE report recommended 72 HZ
conventional television monitors. Computer monitors are
(for 24 fps ?lm material) and 75 HZ (for 25 fps ?lm material)
typically progressively scanned (i.e., non-interlaced) and use
display refresh rates for the presentation of video images, it
provided no guidance regarding implementation of an image
display and input system that Would accommodate the
relatively high scan refresh. Most computer-type monitors
are capable of displaying a Wide range of refresh rates from
60 HZ upWard. HoWever, relatively high refresh rates (typi
recommended display rates. Nor is there any description in
cally exceeding 70 HZ) are used to avoid the Well-knoWn
the prior art that teaches or suggests a computer-compatible
“?icker” effects and eyestrain that occurs When 60 HZ
image display and input system that provides computer
refresh rates are used.
compatible fully synchroniZed video and audio information
[0016] Many attempts have been made during the recent
past to incorporate audio and video information into the
personal computer and Workstation environment. Due to the
inherent incompatibility of the different information types,
at the recommended display rates.
[0020] As is Well knoWn, the prior art approach of dis
playing video information on computer-type monitors typi
cally involved dropping or repeating frames because the
this integration typically has required several processing
computer-type monitor screen refresh rate is not synchro
steps, With trade-offs in video quality, cost, functionality,
poWer, etc. The problems associated With providing video
niZed With the image update rate. HoWever, a problem With
the prior art techniques has been that, in the absence of
information for use With computers are Well documented as
special processing, video signals displayed on computer
exempli?ed by a text entitled “Video Demysti?ed”, Written
type monitors can have psycho-visually disturbing motion
by Keith Jack, and published in 1993 by Hightext Publica
discontinuities due to the differences in the source video and
monitor frame rates. As is Well knoWn, and as described in
tions, Inc., Solana Beach, Calif. (referred to hereafter as “the
Jack text”). The Jack text is incorporated by reference in
their entirety herein for their teachings on video image
processing.
[0017] Exemplary prior art attempts at providing video
information for use With computers are disclosed in the
following US. patents: US. Pat. No. 6,118,486 (issued to
Reitmeier on Sep. 12, 2000 for “Synchronized Multiple
Format Video Processing Method and Apparatus”); and US.
Pat. No. 6,222,589 (issued to Faroudja, et al., on Apr. 24,
2001 for “Displaying Video on High-Resolution Computer
Type Monitors Substantially Without Motion Discontinui
ties”). Both exemplary prior art US. patents are incorpo
rated herein by reference in their entirety for their teachings
on video image processing. As described in the incorporated
the incorporated Jack text, “tearing” of a video image can
occur When the frame rate of a video source is not synchro
niZed to a graphics display. Video frame buffers are usually
doubled-buffered to implement simple frame-rate conver
sion and avoid tearing of the video picture. Simple inex
pensive techniques of displaying video information on com
puter-type monitors can result in degradation of picture
resolution and in the production of intermittent double
images (or dropped images) that may be visible. Techniques
causing feWer disturbing processing artifacts, such as frame
interpolation, are typically very complex and expensive to
implement.
[0021] Therefore, a need exists for a method and apparatus
that overcomes the disadvantages associated With the prior
references, different digital imaging techniques and systems
art techniques and provides fully synchroniZed audio/video
have been proposed facilitating the use of video imagery in
computers. The digital imaging systems vary in terms of
complexity, design, functions, etc. Due to the variations in
information for use With computers and for display on
computer-compatible monitors. A need exists for a display
and input system that includes a computer interface, and that
Feb. 27, 2003
US 2003/0038807 A1
automatically controlled by the inventive display and input
provides synchronized digital video and audio input, as Well
as synchronized digital and analog audio/video output. In
addition to providing fully synchronized audio/video output
system. Thus, a computer need only interact With the inven
tive system via a single buffer request for each frame at the
at the common television display rates of 50 Hz and 60 Hz
24 fps frame rate. Similarly, for 36 fps and 37.5 fps video,
interlaced and non-interlaced, the display and input system
should provide synchronization of moving images for dis
play on 72 Hz and 75 Hz computer-compatible monitors.
The display and input system should facilitate 72 Hz and 75
Hz display of synchronized moving images, such as 24 fps
tripled on 72 Hz computer-monitor displays, Without using
the prior art frame dropping, uneven (intermittent) frame
repeating, or tearing techniques.
a “double-repeat” of a given frame stored Within the frame
buffer is automatically controlled by the display system. The
“double-repeat” method is used to provide synchronized
display of 36 fps and 37.5 fps images on 72 Hz and 75 Hz
displays, respectively.
[0026] In one embodiment, existing horizontal blanking
intervals are reduced to provide increased frame rates and
increased pixel counts of images conveyed on digital inter
The present invention provides such a method and
faces. For example, in accordance With one aspect of the
apparatus for inputting and displaying fully synchronized
present invention, a 1280 (pixels) horizontal by 720 (lines)
vertical formatted digital image is provided at 72 fps using
[0022]
audio/video information for use by computers.
a standard digital video interface. In another embodiment,
SUMMARY OF THE INVENTION
[0023] An inventive display and input method and appa
ratus providing fully synchronized audio/video information
instead of reducing the horizontal blanking intervals (also
referred to as the “retrace time”), the pixel and data rate
clocks are proportionally increased to produce 72 fps and 75
fps video formats. In yet another embodiment of the present
for use With computers and for display on computer-com
invention, pixel and data rate clocks are reduced by a
patible monitors is described. The inventive display and
input system includes a computer interface, and provides
synchronized digital video and audio input, as Well as
1000/1001 reduction factor to support compatibility With
legacy NTSC and other 59.94 Hz video systems.
synchronized digital and analog audio/video output. In addi
tion to providing fully synchronized audio/video output at
the common television display rates of 50 Hz and 60 Hz
interlaced and non-interlaced, the display and input system
provides synchronization of moving images for display on
72 Hz and 75 Hz computer-compatible monitors. The inven
tive display and input system facilitates 72 Hz and 75 Hz
display of synchronized moving images, such as 24 fps
tripled on 72 Hz computer-monitor displays, Without using
frame dropping, uneven frame repeating, or tearing tech
niques. By taking advantage of unused bandWidth available
in the blanking intervals used by existing interfaces, the
inventive display and input system permits increases in
resolution capacity of 24 and 25 fps images using existing
interfaces.
[0027] The inventive display and input system ensures full
synchronization of both digital and analog audio/video
information. In one embodiment, the present inventive dis
play and input system supports digital and analog video
inputs having one of tWo selected synchronization modes. In
a ?rst video input synchronization mode supported by the
present invention, the analog or digital video source devices
input a video input data rate to the inventive system. In a
second video input synchronization mode, the present inven
tive display and input system outputs the data rate signal to
the video source devices. In both cases, the display and input
system ensures synchronization by interlocking the external
data rate and pixel rate clocks to internally-generated clock
ing signals. One exemplary clock synchronization method
makes use of Well-knoWn phase-locked loop techniques that
lock the external and internal clocks to one another. Alter
[0024] One embodiment of the present display and input
native clock synchronization techniques may be used to
system uses a frame buffer memory for the storage of video
practice the present invention.
[0028] The details of the preferred and alternative embodi
frames. In one exemplary embodiment, the buffer memory is
organized as a “FIFO-of-frames” (or “FIFO-of-display buff
ers”), Wherein video frames are input to the frame buffer
memory on a ?rst in, ?rst out basis. In this embodiment, the
unit of buffering used by the buffer memory comprises a
video frame. In this embodiment, a relatively large number
of video frames can be stored in the frame buffer. As long as
tWo or more frames can be stored Within the buffer memory,
ments of the present invention are set forth in the accom
panying draWings and the description beloW. Once the
details of the invention are knoWn, numerous additional
innovations and changes Will become obvious to one skilled
in the art.
BRIEF DESCRIPTION OF THE DRAWINGS
automatic display synchronization With respect to image
[0029]
frame rates can be achieved.
embodiment of a fully synchronized display and input
[0025]
In accordance With one aspect of the present inven
FIG. 1 shoWs a block diagram of an exemplary
system made in accordance With the present invention.
tion, a “triple-repeat” method (of 24 fps and 25 fps video) is
[0030]
used to provide synchronized display onto 72 Hz and 75 Hz
implementation of the fully synchronized display and input
displays, respectively. In accordance With this method, the
triple-repeat of video frames (stored Within the frame buffer
memory) alloWs 24 fps images to be synchronized With
display refresh rates. This synchronization is achieved by the
present inventive system With very little computer interac
system of FIG. 1.
[0031] Like reference numbers and designations in the
various draWings indicate like elements.
tion. In accordance With this inventive aspect of the inven
tion, selected frames are thrice repeated (i.e., selected frames
are output from the frame buffer memory) during frame
buffer memory accesses. The triple-repeat of video frames is
FIG. 2 shoWs a block diagram of an exemplary
DETAILED DESCRIPTION OF THE
INVENTION
[0032] Throughout this description, the preferred embodi
ment and examples shoWn should be considered as exem
plars, rather than as limitations on the present invention.
Feb. 27, 2003
US 2003/0038807 A1
[0033]
FIG. 1 shows a block diagram of one exemplary
embodiment of a fully synchronized display and input
system 100 made in accordance With the present invention.
As described below in more detail, in addition to performing
other important functions, the inventive display and input
system 100 facilitates the display of synchroniZed moving
images at display refresh rates of 72 HZ and 75 HZ Without
frame dropping or tearing. The inventive display and output
system also ensures that the audio is fully synchroniZed With
associated video. As described in more detail beloW, the
display and input system 100 provides synchroniZed 24 fps
images and audio for display on 72 HZ displays (i.e., 24 fps
tripled on 72 HZ displays). The inventive display and input
system 100 similarly provides synchroniZed 25 fps images
and audio for display on 75 HZ displays. The 72 HZ and 75
HZ moving image and audio information is computer com
patible and therefore accessible by a computer (or other
digital processing device) via a digital interface. Other frame
one embodiment, the digital video input 130 receives digital
video conforming to the incorporated SMPTE 292M Bit
Serial Digital Interface standard. Although some embodi
ments of the present invention are described beloW With
reference to the incorporated SMPTE 292M standard, it Will
be understood that the scope of the present invention is not
limited to use With any particular digital interface, and that
the present inventive display and input system can be used
With any convenient or useful digital video interface.
[0039] In one embodiment of the present invention, the
system 100 derives a clock signal from the digital video
input 130 in order to permit the contemporaneous display of
video and audio information While the video information is
being input to the system 100. Alternatively, the digital
video input 130 may be buffered into the system 100 Without
display on a display device. In this case, as long as all video
information is acquired from the digital video input 130
rates are also accommodated by the present inventive system
Without any loss of data, the system 100 does not require
100. For example, moving images at 36 fps and 37.5 fps are
synchroniZed by the system 100 and displayed on 72 HZ and
75 HZ displays. The synchroniZed audio/video information
[0040] Optional Analog Video Input—A/D Converter
may be output for display to a computer-type monitor or
[0041] As shoWn in FIG. 1, analog video information may
also be optionally input to the system 100 via the A/D
other display device.
locking to the incoming digital video input data clock.
converter block 106. The A/D converter block 106 is capable
[0034]
Referring noW to FIG. 1, in one exemplary
embodiment, the synchroniZed display and input system 100
includes the folloWing video interfaces: an optional video
input block 102 including a digital video input interface 104
and analog-to-digital
converter 106, an optional ana
log video output block 108 including a random access
memory digital-to-analog converter (RAMDAC) 110, and
an optional digital video output block 112 including a digital
video output interface 114. The exemplary embodiment 100
also includes an optional audio input/output 116 comprising
an audio input block 118 and an audio output block 120. The
exemplary synchroniZed display and input system 100 also
includes a buffer memory 122, a color space transform block
124, and a clock synchroniZation system 126 including
clock synchroniZation circuitry. The inventive display and
of receiving any of the Well-known analog video input
signals including RGB, YCrCb, YC (also knoWn as
S-video), for example. The A/D converter block 106 may be
implemented using any commercially available ADCs
capable of digitiZing analog video information. In one
embodiment, the A/D converter block 106 is capable of
sampling at rates of 10 to 150 million samples per second
(MSPS). In accordance With one aspect of the present
invention, and as described in more detail beloW, the system
100 generates a harmonic of the horiZontal scan rate When
inputting analog video. In one embodiment, as described in
more detail beloW, the horiZontal scan rate harmonic is
produced using a harmonic phase locked loop (PLL) circuit.
A pixel clock signal is thereby derived from the horiZontal
scan rate harmonic and used by the A/D converter block 106
input system 100 also includes a computer interface 128.
The computer interface permits access to the buffer memory
When sampling analog video input to the A/D converter
block 106 via an analog video input 132.
122 by a computer (not shoWn).
[0035] Each of the blocks of the exemplary display and
[0042] Analog Video Output
[0043] The inventive synchroniZed display and input sys
input system 100 shoWn in FIG. 1 is brie?y described beloW
in separate sections. A description of hoW the various
components shoWn in FIG. 1 function together to imple
ment the inventive aspects of the synchroniZed display and
tem 100 of FIG. 1 optionally includes an analog video
output block 108 including a RAMDAC 110. The RAM
DAC 110 can be implemented using any Well-known com
input system of the present invention folloWs the description
mercially available RAMDAC device (or RAMDAC func
of the various components.
[0036] Optional Digital Video Input
tional block of a device). The RAMDAC 110 converts
digital pixel values of video images stored Within the buffer
[0037] As shoWn in FIG. 1, the exemplary synchroniZed
display and input system 100 includes an optional digital
and analog video input block 102. The optional video input
memory 122 into an analog video output signal. As shoWn
in FIG. 1, the analog video output is provided at an analog
video output 134. In one embodiment of the present inven
tion, the RAMDAC 110 adds an additional modi?cation of
block 102 includes an optional digital video input interface
104 and an optional analog video interface comprising an
A/D converter block 106. Any Well-known digital video
interface may be used in implementing the digital video
input interface 104. As shoWn in FIG. 1, digital video is
input to the system 100 via a digital video input 130.
in order to change the curve representation of pixel values
With respect to brightness or color. HoWever, unlike the
video transfer function adjustments available Within the
color space transform system 124 (described beloW in more
detail), the RAMDAC does not provide for cross-color
[0038] In one embodiment, the digital video input inter
face 104 accommodates digital component video inputs
using separate color components, such as YCrCb or RGB. In
the video transfer function, Which is often a gamma curve,
terms.
[0044] The optional analog video output block 108 also
provides the horiZontal and vertical sync pulses at the analog
Feb. 27, 2003
US 2003/0038807 A1
video output 134. As is Well known, the horizontal and
vertical sync pulses are required for the display of analog
video images. For example, in an NTSC television system,
a horiZontal sync pulse is transmitted for each horiZontal line
[0050] Audio Input/Output
[0051] In one embodiment, the inventive display and input
system 100 includes an optional audio input/output (I/O)
to keep horiZontal scanning synchroniZed. Similarly, the
block 116 including an audio input block 118 and an audio
output block 120. The audio I/O block 116 provides a
vertical sync pulse is transmitted for each ?eld to synchro
mechanism for inputting (and outputting) audio information
niZe the vertical scanning motion. The synchroniZing pulses
are typically transmitted as part of the picture signal but are
sent during the blanking periods When no picture informa
tion is transmitted. In the embodiment shoWn in FIG. 1, the
horiZontal and vertical sync pulses are typically derived as
sub-multiples of the pixel clock. In one embodiment, these
pulses are produced by the clock synchroniZation system
126 (described in more detail beloW) and output via the
to (or from) the system 100. The audio may be analog or
digital.
[0052]
As described in more detail beloW, the present
inventive synchroniZed audio/video display and input sys
tem 100 ensures that audio information is fully synchroniZed
With its associated video information. If the audio and video
are in a digital format, synchroniZation is achieved by
analog video output 134.
requiring that the audio and video clocks be locked to one
another. If a digital interface conforming to the above
[0045] As described beloW in more detail, in one embodi
ment of the present invention, the video vertical rate is 72
incorporated SMPTE 292M standard is used (Which is
capable of transmitting embedded audio in some formats),
the digital audio and video information may be input from
the same digital interface. HoWever, if separate digital
HZ, 75 HZ, or their 1000/1001 variants. The picture update
rate may comprise 24, 36, or 72 frames per second, or 25,
37.5, or 75 frames per second, or the 1000/ 1001 variants of
these picture update rates.
interfaces are used to generate the digital audio and video
information, the separate audio and video pixel sample
clocks are interlocked at the audio/video source. For
[0046] Digital Video Output Interface
[0047] As noted above, the exemplary embodiment of the
synchroniZed display and input system 100 of FIG. 1
optionally includes a digital video output block 112 having
a digital video output interface 114. The optional digital
video output block 112 provides a digital video output signal
on digital video output 136. The digital video output signal
carries digital video data for input to a digital video pro
cessing, storage, or display device (not shoWn). The digital
video output 136 may conform to any convenient digital
video interface speci?cation. For example, the digital video
output 136 may be interfaced to a device that accepts Digital
Video Interactive (DVI) digitally-formatted data. As another
example, the digital video output 136 may conform to the
above-incorporated SMPTE 292M Bit-Serial Digital Inter
face for High-De?nition Television Systems standard. As
noted above, although the present invention is described
With reference to the incorporated SMPTE 292M standard,
it Will be understood that the scope of the present invention
is not limited to use With any particular digital video
interface.
[0048] In some embodiments, the digital video output
example, When a high de?nition formatted display is used,
such as, for example, a 1280 (pixels) by 720 (lines) 72 fps
video display, the SMPTE 292M standard does not have
room to accommodate embedded audio in this format.
Therefore, separate digital interfaces are required for the
digital audio and video information, and the audio and video
sample clocks are interlocked at the audio/video source.
Alternatively, the audio and video sample clocks are inter
locked via a clock ladder in the present inventive input
system.
[0053] When analog audio is used, the audio sample rate
With digital video input is derived using an audio input
clock. The audio input clock is derived by the system 100
from the digital video pixel or input rate. When analog audio
is used together With an analog video input (Wherein the
video is provided via the analog video input 132 described
above), the system 100 uses a pixel clock, Which is derived
from the horiZontal scanline rate signal, to synchroniZe the
audio information With the video information. In one
embodiment, the pixel clock is derived as a phase-locked
loop harmonic of the horiZontal scanline rate and is used as
the source of the derived analog audio. When digital audio
is used together With analog video, the pixel clock is derived
interface 114 outputs a digital video output 136 having a
as a harmonic of the horiZontal video rate. The video sample
video frame rate of either 72 or 75 frames per second. In
rate is thereby ensured to be locked to the digital audio
addition, as described beloW in more detail, the digital video
sample rate.
output 136 may use a video frame rate of 72*1000/ 1001 HZ,
or 75*1000/1001 (in order to provide synchroniZation With
the NTSC television standard). The digital video output 136
can be used for display With display devices that accept
digital video signals. The digital video output 136 can be
provided as input to other useful digital video devices such
as recorders, sWitchers, processors, and any other useful
device that processes or stores digital video information.
[0049]
As is Well knoWn, and as described in the incor
porated SMPTE 292M standard, digital video interfaces
typically transmit a pixel clock together With the actual pixel
data values. Although the pixel clock of some display and
digital video processing devices may differ from the digital
video interface clock, the clocks typically are locked to one
another.
[0054] Optional Color Space Transforms
[0055] As is Well knoWn and as described in the incorpo
rated Jack text, color information can be digitally repre
sented using color spaces. Color spaces comprise math
ematical representations of color information. Many color
spaces can be used in practicing the present invention,
including RGB, YIQ, YUV, Hue Saturation Luminance
(HSL), Hue Saturation Value (HSV), Luminance u‘ v‘, and
others. The Color Space Transform (CST) block 124 option
ally performs input color space transformations on incoming
video (input to the system 100 via the video input block 102)
before it is stored in the buffer memory 122. On output, the
CST block 124 optionally performs output color space
transformation of the stored digital video information before
it is output via either the RAMDAC 110 (i.e., analog video)
Feb. 27, 2003
US 2003/0038807 A1
or the digital video output interface 114 (i.e., digital video).
The CST block is “optional” because in some operating
modes, the CST block 124 performs no color space trans
formation on the digital video information, but rather simply
such optional curve modi?cations can have optional cross
color terms (also knoWn as a color matrix).
[0059] Clock SynchroniZation System
passes the video information through (on input to the frame
[0060] As shoWn in the exemplary embodiment of FIG. 1,
buffer memory, and on output to either the RAMDAC 110 or
the present inventive synchroniZed display and input system
the digital video output interface 114).
100 includes a clock synchroniZation system 126. In one
embodiment, the clock synchroniZation system 126 com
[0056] An exemplary input color space transformation
prises circuitry including phase lock loops (PLL) that syn
performed by the CST block 124 transforms RGB color
chroniZe the various video pixel and audio sample clocks.
The phase lock loops may be implemented in hardWare,
space to YUV color space. Alternatively, the CST block 124
transforms the YUV color space to RGB. As is Well knoWn
softWare, or a combination of both hardWare and softWare.
in the digital video processing arts, in the YUV color space,
it is typical to reduce the U and V chroma resolution either
As described in more detail beloW, clocking signals, such as
horiZontally, vertically, or both vertically and horiZontally.
signals, are derived from a pixel clock. An internal pixel
clock is also provided. In one exemplary embodiment, an
internal 1.485 Gbit/second reference data clock is provided
and used to derive other internal clocks used by the system
100.
The U and V chroma resolution is usually reduced in half,
although other reduction ratios can be used in practicing the
present invention. The reduction of chroma resolution
reduces 0.25 both the memory bandWidth and siZe require
ments associated With the buffer memory 122. The CST 124
also alternatively performs color space transformations from
a ?rst set of RGB primaries to a second set of RGB
primaries. As is Well knoWn, the incorporated SMPTE 292M
standard supports YUV having half horiZontal resolution in
U and V in a single-link mode (i.e., When a single SMPTE
292M serial digital interface is used for the I/O of digital
video). In this case, the color space transform block 124
converts the YUV format to (or from) RGB Within the buffer
memory 122. When a dual-link mode is used (i.e., When a
“dual-link” SMPTE 292M serial digital interface is used for
the I/O of digital video), full resolution U and V, as Well as
full resolution RGB, are also supported by the system 100.
RGB plus Alpha color spaces can also be supported to
the horiZontal scan rate, frame rate, and vertical scan rate
[0061] HoWever, those skilled in the digital video and
audio processing arts shall recogniZe that other convenient
reference clocks can be used Without departing from the
scope or spirit of the present invention. The internal pixel
clock is used When there is no external video input (i.e.,
When there is no incoming video signal provided at the
digital video input 130). As described in more detail beloW,
the internal pixel clock generated by the clock synchroni
Zation system 126 is used When video is displayed or output
via the analog (134) or digital (130) video outputs and When
not simultaneously inputting video. In other related embodi
ments, pixel clock rates such as 74.25 MHZ or the related
89.1 MHZ may be used as the internal reference.
provide a composite matte signal for production input appli
[0062] The clock synchroniZation system 126 communi
cations.
cates With other components of the inventive system 100 to
[0057] An exemplary output color space transformation
performed by the CST block 124 transforms YUV color
space to RGB color space. Such a color space transformation
is particularly useful because most computer-type display
devices utiliZe RGB signals. In addition, When performing
the YUV to RGB color space transformation, the CST block
124 optionally increases chroma resolution in U and V. The
provide the internal clocking signals to the various process
ing blocks. For example, as shoWn in FIG. 1, the clock
synchroniZation system 126 provides clocking signals to the
optional audio I/O block 116, the buffer memory 122, the
CST 124, and the video output blocks 108, 112 via a
plurality of clock/control lines 144. Details regarding the
inventive aspects of the clock synchroniZation system 126
are described beloW in more detail With regard to the
increase in chroma resolution may be performed vertically,
horiZontally, or both horiZontally and vertically. Many other
color space transformations are possible. For example, in
description of synchroniZation (by the inventive system 100)
to video (and audio) input and output devices.
one embodiment, as noted above, the CST 124 may perform
RGB to RGB color space transformations. Such transfor
mations may be useful When using video displays or video
[0063] Buffer Memory
output devices that require color primaries other than those
used by the RGB pixels stored Within the buffer memory
122. Also, When processing SMPTE 292M formatted video
(or other digital video standards that operate using YUV
formats), the CST 124 can be used to convert from RGB (or
other formats) to the digital YUV format needed by the
digital video output interface block 112 (or for storage in the
frame buffer memory 122).
[0058] In addition to performing input and output color
space transformations, the CST 124 can also change the
video transfer function. As is Well knoWn, the video transfer
function often comprises a gamma curve. The CST 124 can
modify the video transfer function in order to change the
curve representation of pixel values With respect to bright
ness and color. Within the color space transform block 124,
[0064]
The buffer memory 122 stores frames of digital
video in a selected color space. In one embodiment, the color
space is determined by the color space transform system 124
for use With video I/O. In another embodiment, the color
space used in storing the video information Within the buffer
memory 122 is selected by a computing device (not shoWn)
that interfaces With the system 100 via a computer interface
128.
[0065] Any convenient number of video frames can be
stored Within the buffer memory 122.
[0066] The siZe of the buffer memory varies in accordance
With system requirements. In one embodiment, the memory
122 is structured as a “First-In, First-Out” (FIFO) memory,
Wherein the input and output of the buffer memory 122 are
independently clocked. An exemplary embodiment of the
buffer memory 122 uses the Well-known “ring buffer” orga
Feb. 27, 2003
US 2003/0038807 A1
nization. Alternatively, any other suitable or convenient
buffer memory organization can be used to implement the
FIFO buffer memory structure. In one embodiment, the
buffer memory comprises 128 Mbytes, although larger and
smaller memory sizes can be used to practice the present
invention.
[0067] In one exemplary embodiment of the present
inventive display and input system 100, the buffer memory
122 is organized as a “FIFO-of-frames” or “FIFO-of-display
buffers”, Wherein video frames are input to the buffer
memory 122 on a ?rst-in, ?rst out basis. In this embodiment,
the unit of buffering used by the buffer memory 122 com
prises a video frame (possibly also including associated
audio as described beloW in more detail). In this embodi
ment, a relatively large number of frames can be stored in
the FIFO frame memory (for example, approximately 50
frames can be stored at 1280x720 resolution using YUV
formatting, With half U and V horizontal resolution). As
described beloW in more detail, as long as tWo or more
frames can be stored Within the buffer memory’s FIFO
con?guration, automatic display synchronization With
respect to image frame rates can be achieved.
[0068] The FIFO organization (and independent I/O
clocking) of the buffer memory 122 ensures that variations
in the rate of computer-implemented digital video process
ing does not adversely affect the display of images stored in
the buffer memory 122. Stated in another Way, the FIFO
organization (and independent I/O clocking) used to imple
ment the buffer memory 122 provides timing “slack” to the
computing device (or devices) coupled to the computer
interface 128. The timing slack alloWs the system 100 to
support multiple digital video processing functions. The
time required to perform each digital video processing
function varies depending on the function. As is Well knoWn
in the digital video processing arts, variations in digital
video processing timing are due to many common process
ing variables inherent to computer video processing soft
Ware. Using the FIFO-structured buffer memory 122, and
independent I/O clocking of the memory, these video pro
cessing variations do not degrade synchronization perfor
converter 106), and for outputting the stored digital video
frames upon request to a computing device (via the com
puter interface 128). Alternatively, the buffer memory 122 is
dedicated for video output only (i.e., the entire buffer
memory 122 is dedicated for outputting digital video, either
through the digital video output interface 114 or the RAM
DAC 110). In yet another embodiment, the buffer memory
122 is partitioned into tWo sections, Which can be equal or
unequal in size, one for the input of video frames, and the
other for the output of video frames.
[0071] Computer Interface
[0072] As shoWn in FIG. 1, digitized video data is
exchanged betWeen a computing device (not shoWn)
coupled to the computer interface 128 using the data bus
lines 138 (output data bus) and 140 (input data bus). In one
embodiment of the present invention, the computer interface
comprises the Well-knoWn 64-bit PCI-Bus interface. HoW
ever, this computer interface is exemplary only and is not
meant to limit the scope of the present invention. Those
skilled in the digital processing and computing arts shall
recognize that any convenient and suitable computer inter
face can be used to practice the present invention, provided
that the interface supports required video data transfer rates,
and provided that the interface supports required control
registers and data clocks.
[0073] As shoWn in FIG. 1, a synchronization ?ag control
signal is provided as input to the computer interface 128, and
as input to the computing device (not shoWn), via a control
signal line 142. The synchronization ?ag control signal
indicates availability of the buffer memory 122 (as described
above) to the computing device. Buffer availability indica
tion is provided for each of the input and output functions of
the buffer memory 122. Buffer availability indication and
display synchronization techniques are noW described in the
folloWing section.
[0074] Display (Output) Buffering and Synchronization
[0075] In one embodiment of the present invention, dis
play synchronization With respect to image frame rates is
mance of the inventive display and input system 100. Nor do
achieved by utilizing tWo or more buffered frames or frame
they adversely affect the display of video information stored
buffers, and using an automated hold-off system for access
ing the frame buffers. When frame buffers are available,
images are loaded into the buffer memory 122 as the images
are made available by a computer (not shoWn in FIG. 1).
HoWever, When all of the frame buffers are full, Which
in the memory 122.
[0069]
Use of a large number of frames aids in smoothing
variations in the frame transfer rate from the computer. For
example, When playing images from a disk system (such as,
for example, a RAID disk array), the FIFO-of-frames buffer
memory 122 can mask disk seek latencies that are occasion
occurs When active synchronized display (possibly includ
ing “repeat frame displays” described beloW in more detail)
of the images is still in progress for the frame buffer that
Would be available next, the system 100 signals the com
ally required. In addition, the FIFO frame memory aids in
smoothing variations in the time required to decode each
puter using the synchronization ?ag (via control line 142)
frame during realtime-synchronized decompression. Some
that no frame buffer is currently available.
frames (such as, for example, I or B frames) may take longer
to decompress than do other frames. It is valuable to be able
to mask these occasionally sloWer frames or sections of
frames using a sufficiently large buffer memory 122. In this
manner, perfect synchronization of moving image displays
[0076] When the computer is signaled that no frame buffer
is currently available, the computer either Waits until a frame
buffer is available, or it performs other tasks. In accordance
With this aspect of the present invention, the computer Waits
is maintained over arbitrarily long time periods.
until the inventive display and input system 100 signals the
computer (either via an interrupt signal (such as the syn
[0070] In one embodiment of the present inventive display
and input system 100, the entire buffer memory 122 is
chronization ?ag) or via a status register that is accessible to
dedicated for digital video input (i.e., the entire buffer
memory 122 is dedicated to inputting digital video, either
of the needed frame is complete, and that the associated
from the digital video input interface 104 or the A/D
is therefore available for use by the computer.
the computer via the computer interface 128) that the display
frame buffer (previously used to store the displayed frame)
Feb. 27, 2003
US 2003/0038807 A1
[0077] As described below in more detail, one embodi
ment of the present invention uses a “triple-repeat” method
24 fps images to be synchronized With the display refresh
[0082] In one exemplary embodiment, as described above,
the buffer memory 122 comprises 128 Mbytes of FIFO
organized frame memory. In one embodiment of the present
inventive system 100, as noted above, the entire buffer
memory is dedicated to display (or output) of the video
frames stored in the buffer memory 122. Alternatively, the
rate With a minimum of computer interaction. The triple
buffer memory 122 can be dedicated to input of video frames
repeat method is automatically controlled by the display
system 100. Thus, the computer need only interact With the
(sourced from the optional video input block 102, for
example). In yet another embodiment, the buffer memory
122 is partitioned and shared betWeen input and display
(of 24 fps and 25 fps video) to provide synchronized display
onto 72 Hz and 75 Hz displays, respectively. The use of
triple-repeat of a frame Within the buffer memory 122 alloWs
display system via a single buffer request for each frame at
the 24 fps frame rate. Similarly, for 36 fps (and 37.5) fps
(output) of digital video. When the buffer memory 122 is
video, a “double-repeat” of a frame Within a frame buffer is
partitioned (for example, partitioned With half of the buffer
automatically controlled by the display system 100. Conse
memory 122 dedicated for input buffering, and half for
output buffering), computer video processing can be per
quently, computer interaction need only occur When a frame
buffer is requested at the 36 fps rate. The inventive frame
repetition method is described in more detail beloW in a
separate section.
formed on the input video stream While the same input video
stream (or another selected video stream, or a processed
[0078] Input Buffering and Synchronization
version of the selected input video stream) is contempora
neously output (and/or displayed). The ?exible use of a large
buffer memory 122 is bene?cial in alloWing computer
[0079] As With output synchronization, the present inven
systems to support synchronized input, synchronized output
tive display and input system 100 uses the FIFO-structured
buffer memory 122 to synchronize input of video data to the
display) of video images.
computer (coupled to the computer interface 128). As
described beloW in more detail, input synchronization
betWeen the system 100 and the computer is based upon the
availability of frame buffers in the buffer memory 122. As
shoWn in FIG. 1, a description of hoW the various compo
noted above, in one embodiment, the buffer memory 122
uses a FIFO-of-frames con?guration. In this con?guration,
the buffer memory 122 is organized into buffered frames
(also referred to herein as “frame buffers”), Wherein the
buffered frames or frame buffers are accessible on a First-In,
First-Out basis, and Wherein the buffered frames each con
tain one frame of digital video information.
[0080] Similar to the synchronization mechanism
described above With reference to output buffering, if the
computer’s video frame input rate exceeds the display and
input system’s video data rate (i.e., the rate at Which the
present inventive display and input system 100 stores video
frames in the buffer memory 122), the display and input
system 100 signals the computer that a buffered frame is not
yet available. When the next requested input buffered frame
is completely received by the display and input system 100
and stored Within the buffer memory 122, the system 100
signals the computer that the requested buffered frame is
available for input. The inventive display and input system
100 signals the computer via the computer interface 128
described above With reference to FIG. 1.
[0081] In one embodiment, the system 100 signals the
computer that the requested buffered frame is available for
input using an interrupt signal. Additionally, or alternatively,
the system 100 signals the computer that the requested
buffered frame is available for input to the computer by
setting a status bit in a register (or ?ip-?op device) acces
sible to the computer via the computer interface 128. As long
as the buffer memory 122 is capable of holding at least tWo
video frames, the computer can transfer video frames from
a ?rst frame buffer While the display and input system 100
inputs the next video frame to a second frame buffer. If many
frame buffers are available in the buffer memory 122,
variations in the computer’s ability to accept or process the
video frames can be smoothed so that synchronization
betWeen the computer and the inventive system 100 is
maintained.
and/or display, or both simultaneous input and output (or
[0083] Having described the various component blocks
nents function together to implement the inventive synchro
nized display and input system is noW provided. As
described beloW in more detail, the present inventive
method and apparatus provides a facility for achieving 72
Hz and 75 Hz computer display, on the input, output, of
synchronized moving images, such as 24 fps tripled on a 72
Hz display, and 25 fps tripled on a 75 Hz display, Without
frame dropping or tearing of the moving images. The
inventive method and apparatus also provides fully synchro
nized audio (Wherein the audio information is fully synchro
nized With associated video). Fully synchronized 72 Hz and
75 Hz video/audio information is stored in the buffer
memory 122 for output (on a display device and/or audio
device) and/or input to a computing device. In another
aspect of the present inventive method and apparatus,
improvements in resolution capacity of 24 fps video are
made using unused available bandWidth present in the
blanking intervals of existing digital video interfaces. Other
aspects and features of the present inventive method and
apparatus are described beloW.
[0084] Reducing SMPTE 292M Horizontal Blanking
Intervals to Yield 72 Hz Synchronized Frame Displays
[0085] In one embodiment of the present invention, exist
ing horizontal blanking intervals are reduced to provide
increased frame rates and increased pixel counts of images
stored Within the inventive system 100. For example, in
accordance With one aspect of the present invention, a 1280
(pixels) horizontal by 720 (lines) vertical formatted digital
image is conveyed at 72 fps using an SMPTE 292M digital
video interface.
[0086] As described in the above-incorporated SMPTE
292M high de?nition (HD) digital interface standard, the
HDTV digital interface (and its variants) transmit and
receive YUV (half horizontal U and V resolution) 10-bit
digital video signals for several HD formats. These HD
formats include 1920 horizontal><540 vertical resolution
interlaced ?elds at 60 fps, 1920 horizontal><1080 vertical at
24 fps, and 1280 horizontal><720 vertical at 60 fps. In one
Feb. 27, 2003
US 2003/0038807 A1
embodiment of the present invention, the digital image
via a SMPTE 292M-comforming digital interface. The
stored Within the FIFO-of-frames buffer memory 122 con
present invention uses the above-described “excess” blank
ing information inherent to the SMPTE standard to convey
forms to the proposed SMPTE 296M standard entitled
“1280x720 Scanning, Analog and Digital Representation
and Analog Interface”, published by the SMPTE for com
ments, and incorporated herein by reference for its teachings
on high resolution image formats. As described in the
incorporated SMPTE 296M standard, a family of raster
scanning systems exists for the representation of stationary
or moving tWo-dimensional, images sampled temporally at
a constant frame rate and having an image format of
1280x720 and an aspect ratio of 16:9. In addition to other
aspects, the standard speci?es R‘G‘B‘ encoding, R‘G‘B‘ ana
log and digital representation, Y‘P‘BP‘R color encoding (also
knoWn as YUV) (including analog representation and analog
interface), Y‘C‘BC‘R color encoding (also knoWn as YUV),
digital representation and digital interface.
[0087] As described in the incorporated SMPTE stan
dards, the digital interface and its variants use a 74.25 MHZ
additional active pixel information, via the interface. The
present invention reduces the number of horiZontal blanking
pixels clocked across the digital interface, and uses the
available pixel clocks to convey active pixels.
[0091] For example, in one application of the inventive
horiZontal blanking interval reduction technique, the present
inventive system 100 uses 1375 total pixels (horiZontal) by
750 total lines, at the 74.25 MHZ pixel rate, to produce
1280x720 72 fps digital video. In this embodiment, When
1375 total pixels are used, the blanking interval is reduced
from 370 to 95 pixels (i.e., the blanking interval comprises
1375 (total horiZontal pixels)—1280 (active horiZontal pix
els)=95 pixels). More speci?cally, the present invention uses
the folloWing horiZontal timing pattern (having reduced
horiZontal blanking intervals) using a 74.25 MHZ pixel
clock to transceive 1280x720 72 fps video: horiZ_front
pixel clock to transmit and receive the YUV digital video
signals. As described beloW in more detail, both active pixel
and blanking interval data are transceived via the digital
_porch=5, horiZ_sync_Width=38, horiZ_back_porch=52,
interface. As a consequence, bandWidth is available for the
transmission of additional active video information Within
the otherWise unused blanking interval. In accordance With
one aspect, this available additional bandWidth is used by the
MHZ pixel clock is 5+38+52+1,280, or 1,375. In this
embodiment, the vertical timing pattern is identical to the
vertical timing pattern described above With reference to the
SMPTE standard (i.e., comprising 750 total lines and 30
present inventive synchroniZed display and input method
lines for providing vertical blanking information).
and apparatus to increase the frame rate or pixel counts of
[0092] The horiZontal blanking intervals are thereby
advantageously reduced to relatively short durations using
the inventive method and apparatus. The reductions to the
horiZontal blanking interval permits an existing SMPTE
the above-identi?ed formats. The present inventive method
and apparatus uses a higher proportion of available sample
times to convey active pixel information.
[0088] More speci?cally, the incorporated SMPTE 296M
standard de?nes hoW a 1280x720 formatted video image at
60 fps is provided over a bit-serial interface conforming to
the incorporated SMPTE 292 standard. As de?ned by the
incorporated references, the 74.25 MHZ pixel clock trans
mits 1650 total pixels><750 total lines. The 1650><750
“image” includes both active pixels and blanking informa
tion (both vertical and horiZontal blanking). The folloWing
horiZontal timing pattern is de?ned in the incorporated
standard: horiZ_front_porch=70 pixels, horiZ_sync_Width=
40 pixels, horiZ_back_porch=260 pixels, and horiZ_active
_pixels=1280 pixels. Therefore, the total horiZontal pixel
information transmitted by the 74.25 MHZ pixel clock is
70+40+260+1280, or 1,650 pixels, Which includes both
active pixels and horiZontal blanking information.
[0089] The folloWing vertical timing pattern is de?ned in
the incorporated standard: vert_front_porch=5, vert_sync
_Width=5, vert_back_porch=20, and vert_active_lines=720
lines. Thus, the total vertical line information transmitted by
the pixel clock is 5+5+20+720, or 750 total vertical lines.
Similar to the horiZontal pixel information, the vertical line
information includes both active lines and blanking infor
mation. Thus, the SMPTE standard blanking intervals com
prise 370 horiZontal blanking pixels [1650 (total pixels
transmitted over the interface)—1280 (active horiZontal pix
els)] and 30 vertical blanking lines [750 (total lines)—720
and horiZ_active_pixels=1,280 (total horiZontal pixels).
Thus, the total horiZontal pixels transmitted by the 74.25
292M bit-serial digital interface to be used When transceiv
ing 1280x720 72 fps video information. The reduced hori
Zontal blanking durations are acceptable for display by most
digital displays and some digital cameras. HoWever, the
reduced blanking durations may be incompatible With some
commercially available analog displays requiring retrace,
such as the common Cathode Ray Tube (CRT) displays. Any
analog monitor that is capable of accepting the inventive
reduced horiZontal sync signals can display the 74.25 MHZ
formatted digital signal. As is Well knoWn, some analog
cameras also utiliZe longer retrace times and therefore might
be incompatible With the present inventive method and
apparatus.
[0093] As Will be appreciated by those skilled in the
computing and video processing arts, the reductions in
blanking pixels given above are exemplary only, and should
not be interpreted as limiting the scope or spirit of the
present inventive method and apparatus. The present inven
tion contemplates use of any convenient and useful blanking
interval, and any reduction of the blanking intervals for
purposes of providing 72 fps digital video via a bit-serial
interface falls Within the scope of the present invention.
[0094] Increasing Pixel and Data Clocks to Yield 72 HZ
and 75 HZ SynchroniZed Frame Displays
[0095] In another embodiment of the present invention,
instead of shortening the horiZontal blanking intervals (also
(active lines)].
referred to as the “retrace time”), the pixel and data rate
[0090] In one embodiment, the present inventive display
and input system reduces the horiZontal blanking interval
(i.e., the number of pixels assigned to the horiZontal blank
ing information) to provide a 72 fps 1280x720 video image
clocks are proportionally increased, (including the normal
generous retrace times) to produce the desired 72 fps and 75
fps video formats. For example, in one embodiment, the
74.25 MHZ pixel clock is increased by a factor of 72/60 (or
Feb. 27, 2003
US 2003/0038807 A1
6/5) to produce the 1280><720 72 fps frame displays. In this
embodiment, the 74.25 MHZ pixel clock is increased to a
frequency of 89.10 MHZ. In another embodiment, the 74.25
MHZ pixel clock is increased by a factor of 75/60 (or 5/4) to
produce the 1280><720 75 fps frame displays. In this embodi
ment, the 74.25 MHZ pixel clock is increased to a frequency
of 92.8125 MHZ.
[0096] The exemplary simple pixel clock multiplication
factors (6/5 for 72 fps, and 5/4 for 75 fps) ease implemen
tation and production of the higher pixel clocks and also
permit all of the clock signals to be easily inter-locked. For
example, in one embodiment as noted above, phase-locked
loop circuits are used to lock the 74.25 MHZ pixel clock to
the increased pixel clock signals. Use of simple pixel clock
multiplication factors facilitates the contemporaneous use of
analog and digital video formats. Thus, analog input and
display can be contemporaneously provided to the digital
input and output circuits. Although the exemplary embodi
ment uses simple pixel clock multiplication factors, it Will be
appreciated and understood by those skilled in the digital
video processing arts that other pixel clock multiplication
factors can be used to practice the present invention, and that
the present invention encompasses use of any convenient or
useful pixel clock multiplication factor.
[0097] Increasing Digital Video I/O BandWidth to Yield 72
HZ and 75 HZ SynchroniZed Frame Displays
[0098] Alternatively, or additionally to the techniques
each multiply and addition exceeds 8 bits When performing
digital ?ltering, the result has a higher quality in terms of
both purity and clarity of color.
[0100]
Reduction of Pixel and Data Rate Clocks to Sup
port Legacy NTSC Video
[0101] As is Well knoWn, it is common practice to reduce
the frame rate of 24 fps video by a factor of 1000/1001 to
provide compatibility With NTSC televisions. The resulting
reduced frame rate video is referred to as 23.98 fps video
(although, more precisely, the frame rate equals 23.976024).
As is Well knoWn, NTSC televisions use a 60 HZ refresh rate
reduced by the 1000/1001 factor (i.e., they operate at 59.94
(more precisely, 59.94006) HZ refresh rates). In accordance
With one embodiment of the present invention, synchroni
Zation With 1000/1001 reductions to 72 HZ is provided for
compatibility With such refresh rate reductions of 24 fps
video. For compatibility With this scenario, 72, 36, and 24
fps can all be reduced in the system 100 using the 1000/ 1001
refresh rate reduction factor. As described beloW in more
detail, if audio information comprises 48 kHZ or 96 kHZ
associated With 23.98 fps video (rather than 24 fps video), it
is sometimes desirable to retain the 23.98 frame rate for
images.
[0102]
The 1000/ 1001 rate reductions of many digital
video formats are de?ned in the incorporated SMPTE 292M
digital interface standard. In accordance With one aspect of
the present invention, the desired variation in refresh rates is
achieved by reducing the 74.25 MHZ pixel clock and its
corresponding data rate clock (Which operates at 1.485 Gb/s)
described above, a “dual-link” SMPTE 292M serial digital
interface is used in one embodiment for the input and output
of digital video. Use of an SMPTE 292M dual-link serial
by the 1000/1001 refresh rate reduction factor. More spe
digital interface doubles the bandWidth and I/O capacity of
the system 100 as compared to the single-link embodiment.
ci?cally, in one embodiment, multiplying the 72 fps rate by
the 1000/1001 reduction factor provides compatibility With
This increased I/O bandWidth can be used by the present
the 23.98 fps image rate. This results in a video frame rate
invention to support the 72 fps and 75 fps digital video
of approximately 71.928072 HZ. In one embodiment, legacy
formats described above. Further, as described above, the
NTSC video compatibility is achieved by multiplying the
74.25 MHZ pixel clock by the 1000/ 1001 reduction factor.
SMPTE 292M dual-link digital interface provides suf?cient
additional capacity capable of transceiving full horiZontal
resolution U and V, or RGB channels. Further, using the
dual-link embodiment of the present display and input
system 100, additional data capacity is provided that can be
used to increase the pixel bit precision beyond the common
10-bit pixel color component value. For example, in some
embodiments, higher pixel precision values can be used,
such as 12-bit (or higher) pixel color component values.
[0099] Therefore, some embodiments of the present
invention permit use of greater than 8 bits for each of R, G,
and B (in RGB formats), or Y, U, and V (in YUV formats)
pixel representation. In one exemplary embodiment
described above, 10-bits are used for transfer on the com
puter interface 128, for storage in the FIFO frame memory
buffer 122, for color transformations by the CST 124, for the
This results in a pixel clock of 74.175842176 MHZ. The
inventive approach is described beloW With reference to
synchroniZation of audio.
[0103] As described above, one embodiment of the
present inventive display and input system 100 uses propor
tionally increased pixel and data rate clocks to produce 72
fps (and 75 fps) video. In one embodiment, the present
invention applies the 1000/1001 reduction factor to the
increased pixel and data rate clocks to support 59.94 HZ
legacy NTSC video output compatibility. For example, as
described above, in one embodiment, the 74.25 MHZ pixel
clock is increased by a factor of 6/5 to yield 1280><720 72 fps
frame displays. In this embodiment, the 74.25 MHZ pixel
clock is increased to 89.10 MHZ. In order to provide
compatibility With 59.94 HZ legacy NTSC video, one
embodiment of the present invention multiplies the resultant
performance of transfer-curve lookups, and for digital video
output or digital-to-analog conversion for analog video
increased pixel clock (e.g., the 89.10 MHZ pixel clock) by
output. For color transformations, the use of more than 8 bits
the 1000/1001 reduction factor.
in the computations for color space conversion (Which is
[0104] Increased Image Resolution
processed in one embodiment of the CST 124 as a matrix
transform) greatly improves picture quality. Further, the use
of more than 8 bits When performing color resolution
?ltering, such as from half-horiZontal or vertical resolution
[0105] Another embodiment of the present invention uti
liZes unused data bandWidth present in the incorporated
SMPTE 292M retrace interval to increase the image reso
greatly improve picture quality. For example, if 6 ?lter taps
lution of 24 and 25 fps video images. This embodiment
extends the 1920 (horiZontal)><1080 (vertical) image at 24
are used for color resolution ?ltering, and the precision of
fps and 25 fps up to 2560 (horiZontal)><1080 (vertical) and
to full-horiZontal or vertical resolution in U and V, can also
Feb. 27, 2003
US 2003/0038807 A1
2048 (horiZontal)><1280 (vertical). For example, the total
pixels generated by the digital video interface for 1920><
cally controlled and processed by the present display and
input system 100 With very little interaction required by the
1080 formatted images at 24 fps on SMPTE 292M operating
at 74.25 MHZ is 2750 horizontal (pixels) by 1125 vertical
(lines). Thus, the horiZontal blanking interval used in trans
buffer memory 122 is organiZed as a FIFO of video frames.
mitting 1920x1080 images comprises 830 pixels (2750 total
pixels-1920 active pixels).
computer. As described above, in one embodiment, the
In this embodiment, the computer outputs video frames
(through the computer interface 128) for storage in the buffer
memory at any desired rate, as long as the computer main
[0106] This blanking interval is far too large to be useful.
Indeed, such a generous blanking interval is Wasteful,
tains an average rate that ensures that the FIFO is never
because 1920x1080 video is intended for use With 30 HZ
[0112]
refresh rates, Where the total pixels comprises 2200, and the
horiZontal blanking is therefore 280 pixels (2200 total
pixels-1920 active pixels). Thus, Within the 2750 total
pixels used in transmitting 1920x1080 formatted images at
24 fps, 2560 pixels can easily be accommodated. Further,
tive triple-repeat technique, the system 100 automatically
2048x1280 video images at 24 fps can be accommodated
using SMPTE 292M interfaces having a 74.25 MHZ pixel
clock by using 2250 total pixels horiZontally by 1350 total
emptied.
In accordance With one embodiment of the inven
repeats output of a frame three consecutive times (i.e.,
contents of a given frame buffer are output to the digital
output (134) or analog output (136), and displayed on a
display device) before the next frame is output. This auto
matic frame repetition is performed by the present inventive
display and input system 100 Without interaction by the
computer. As described above, the computer need only
lines. The factoriZation of the 74.25 MHZ pixel clock, (the
pixel clock de?ned in the incorporated SMPTE 292M digital
interface) comprises 24*33* 56*11. In one embodiment of
interact With the display system 100 via a single frame buffer
request for each frame at the 24 fps frame rate. Similarly, for
the present invention, the total pixel and line counts are
frame repetition), the computer only needs to interact With
the display system 100 using a single frame buffer request
derived using this collection of multiplication factors. Thus,
resolution may be substantially increased at 24 fps utiliZing
these and other common digital interfaces for high de?nition
video.
[0107] Frame Repetition for SynchroniZed Output on 72
HZ and 75 HZ Displays
[0108] One embodiment of the present inventive display
and input system 100 includes an inventive technique for
repeating frames of 24 fps and 25 fps ?lm and video images
for synchroniZed display on 72 HZ and 75 HZ monitors,
respectively. As is Well knoWn in the video arts, 25 fps ?lm
frames are commonly repeated on both ?elds of 50 HZ
PAL-compatible television systems. It is also common to
utiliZe 3-2 pull-doWn to present 24 fps ?lm frames on 60 HZ
NTSC-like television systems. The present inventive display
and input system extends these prior art techniques to
provide for a “triple-repeat” of 24 fps and 25 fps ?lm and
video frames for synchroniZed display on 72 HZ and 75 HZ
displays, respectively. One embodiment of the present
inventive display and input system also applies the “double
repeat” (Without the use of interlace) technique to 36 fps and
37.5 fps images for synchroniZed display on 72 HZ and 75
HZ displays, respectively.
[0109] Just as double and triple frame repetition (or
“frame repeats”) techniques are useful, higher numbers of
frame repeats can be applied to other frame rate video for
synchroniZed display on 72 HZ and 75 HZ displays. For
example, in one embodiment of the present invention,
frames are repeated six times to provide 12 fps and 12.5 fps
on 72 HZ and 75 HZ displays, respectively. In another
embodiment, ?ve-repeats are used to provide 12 fps on 60
HZ displays.
[0110] Although a 75 HZ display refresh rate is commonly
used for computer monitors, the technology for synchroni
Zation of moving images With 75 HZ display screens has not
36 fps (and 37.5 fps) video (accommodated using double
for each frame at the 36 fps frame rate. Similarly to the
triple-repetition method, the double frame repetition is per
formed automatically by the present inventive display and
input system 100. Speci?cally, in this embodiment, the
system 100 automatically tWice repeats the output of a frame
buffer (i.e., contents of a given frame buffer are output
through the digital output (134) or analog output (136), and
displayed on a display device) before the next frame is
output.
[0113] As described above, the buffer memory 122 typi
cally comprises a relatively large memory capable of storing
a relatively large number of frames. For example, in one
embodiment, as noted above, approximately 50 frames of
digital video at 1280x720 resolution (using YUV formatted
data With half U and V horiZontal resolution) can be held by
the buffer memory 122. Although smaller and larger
memory siZes can be used to implement the FIFO-of-frames
buffer memory 122, the buffer memory 122 should be
suf?ciently large to hold at least tWo video frames in order
to accommodate the inventive automatic synchroniZed dis
play of video images described in this section.
[0114] SynchroniZation With Digital and Analog Video
and Audio Inputs
[0115] In one embodiment, the present inventive display
and input system 100 supports digital and analog video
inputs having one of tWo selected synchroniZation modes. In
a ?rst video input synchroniZation mode supported by the
present inventive system 100, the analog or digital video
source device (e.g., a video camera, video tape machine, or
other video source device) inputs the data rate of the video
input to the inventive system 100. In a second video input
synchroniZation mode (described beloW in more detail), the
present inventive display and input system 100 provides the
heretofore been available for use With computers. While 72
HZ display rates on computers are less common than 75 HZ,
data rate signal to the video source device.
72 HZ provides the bene?t of alloWing playback at 24 fps
When utiliZing the methods of this invention.
[0111] In accordance With the present invention, the triple
used (Wherein the source device inputs the data rate clocking
repeat of frames Within the buffer memory 122 is automati
all of its internal clock signals (including pixel, scanline,
[0116] When the ?rst video input synchroniZation mode is
signal to the system 100), the inventive display and input
system 100 locks to the video source data rate, and generates
Feb. 27, 2003
US 2003/0038807 A1
frame rate, and audio clocks) based upon the video source
data rate. In one exemplary embodiment, the clock synchro
niZation system 126 uses Well-known phase-locked loop
(PLL) circuits to lock to the video source data rate signal,
and to generate internal clock signals (such as the pixel rate
clock, frame rate clock, etc.) that are locked to the video
harmonic. Thus, a digital video input provides a much
cleaner sample clock (in addition to providing digital pixel
values). This eliminates errors in A/D sample timing and
errors that occur during A/D conversion.
[0119] As noted above, the present invention also supports
source data rate. Input audio should be buffered to alloW
a second video input synchroniZation mode Wherein the
de-multiplexing, and should have its samples locked inside
display and input system 100 provides the data rate signal
the source.
for use by the video source device. When this video input
[0117]
In one embodiment Wherein the video input com
prises digital video conforming to the incorporated SMPTE
292M digital interface standard (or similar digital interface),
the system 100 uses the 1.485 Gbit/second data rate clock
(de?ned by the SMPTE 292M standard) as the data rate
clocking signal. In this embodiment, all of the audio, frame
rate, and other derivative clock signals are derived from this
externally provided 1.485 Gbit/second data rate clock
(instead of, for example, being derived from the system’s
internal 1.485 Gbit/second reference data clock described
above With reference to the clock synchroniZation system
126). As described above, the external 1.485 Gbit/second
data rate clock is provided to the input system 100 via the
digital video interface block 104 and the digital video input
130. As described above, the SMPTE 292M digital input
carries the 74.25 MHZ pixel clock utiliZing a harmonic at
1.485 GHZ for the serial bit clock. Thus, in the embodiment
synchroniZation mode is used, the video source device (e.g.,
a video camera, video tape machine, or other video source
device) locks to the data rate clock output by the inventive
display and input system. As described beloW in more detail,
the audio clock (via a “Word clock” and Longitudinal Time
Code (LTC)) is typically provided separately from the video
clock in this case. Alternatively, the audio clock may be
provided together With the video clock (or be derived from
the video clock). This aspect of the present inventive display
and input system is described in more detail beloW With
reference to the I/O of audio information (described in the
section beloW).
[0120]
Numerous variations of the above-described clock
locking techniques (i.e., Wherein the clocks used internally
by the inventive system are locked to clocks used by external
video and audio devices) are possible. Those skilled in the
audio/video processing design arts shall recogniZe that a
of the present invention Wherein a digital video input signal
is provided that conforms to the SMPTE 292M speci?cation
(and any analogous digital interface speci?cations), a stable
digital interface clock (or set of related clocks) is used as the
myriad of clock locking techniques are possible, and that all
top of a clock ladder. The clock ladder produces all of the
samples and frame rates should not drift With respect to
incoming video streams and associated audio.
video clock signals used by the system 100 (for example, the
pixel clock, scanline clock, frame rate clock, and audio clock
signals).
[0118] When analog video inputs are provided using the
?rst video format, the analog video sources provide hori
Zontal and vertical synchroniZation (or “scan rate”) signals.
In this embodiment, the system 100 (and more speci?cally,
the clock synchroniZation system 126) derives a pixel clock
based upon the horiZontal and vertical sync signals. For
example, in one embodiment, the system 100 generates the
pixel clock using a phase-locked loop harmonic clock gen
eration circuit. In another embodiment, the A/D converter
106 performs this PLL function thereby generating a digi
tiZed video signal including a pixel clock signal. Alterna
tively, this function can be provided by an external A/D
converter (in Which case a digital video signal as produced
by the A/D converter, is input to the system 100). The
derived pixel clock is used by the system 100 to process
video frames and store (and retrieve) frames in the buffer
memory 122. Phase-locked loop harmonic clock generation
techniques are Well knoWn in the analog video arts. For
example, in a number of analog input systems, the pixel
clock is derived using a PLL tuned to a speci?c harmonic
count of the horiZontal scanline rate. For example, a ?at
panel display having an analog input typically generates
such a clock from the horiZontal sync pulses in order to
of these techniques can be used to practice the present
invention. HoWever, to avoid loss of synchroniZation, no
matter What speci?c clock locking technique is used, audio
[0121] Fully Synchronized Audio
[0122]
As is Well knoWn in the audio/video arts, a valuable
part of most moving image sequences (i.e., video) is its
associated audio information. The present inventive display
and input method and apparatus provides video that is fully
synchroniZed With associated audio information. In one
embodiment, video clocks are locked to associated audio
clocks. For example, in one exemplary embodiment, the
video frame rate clock (used to input or output video frames)
is synchroniZed to the audio sample rate clock using a series
of dividers and phase locked loop circuits. In one embodi
ment, the clock synchroniZation system 126 implements this
PLL function and thereby locks the video frame rate and
audio sample rate clocks.
[0123] In one embodiment of the system 100 described
above With reference to FIG. 1, the 74.25 MHZ pixel clock
is divided by the total number of samples per frame to
produce a frame update rate. For example, in one embodi
ment, a pixel clock is divided by a pre-determined division
factor (in one embodiment, by a division factor of 1546.875,
Which is equal to 12,375/8) to create a 48 kHZ clock used for
audio sampling. Adivide circuit that performs the 12,375/ 16
function (divided into 74.25 MHZ) can be used to generate
clock pixels into their appropriate locations Within the
a 96 kHZ audio sampling clock. When an 89.1 MHZ pixel
display. The present inventive display and input system 100
clock is used (for example, as described above, during the
production of 1280x720 72 fps video frame displays), the
pixel clock is divided using a different pre-determined
contemplates use of any of the Well-known PLL harmonic
clock generation techniques for purposes of deriving the
pixel clock from the analog video sync signals. AWeakness
of using a harmonic of the horiZontal rate is that clock jitter
often occurs due to noise in the PLL that is tuned to the pixel
division factor in order to yield a 48 kHZ audio sample clock.
Speci?cally, in one embodiment, the 89.1 MHZ pixel clock
is divided by 1856.25 (7425/4) to produce a 48 kHZ audio
Feb. 27, 2003
US 2003/0038807 A1
sample clock. In another embodiment, the 89.1 MHZ pixel
clock is divided by (7425/8) to produce a 96 kHZ audio
detail, the resultant video is commonly referred to as 23.98
video (although, more precisely, the resultant frame rate
sample clock.
equals 23.976024 fps). If associated audio comprises 48 kHZ
[0124]
or 96 kHZ, and is associated With 23.98 fps video rather than
24.0 fps video, then it is sometimes desirable to retain the
In one embodiment of the present invention, the 72
fps frame rate is divided by a factor of three to create a 24
fps sub-rate for use With commercially available digital
audio devices that accept the Well-knoWn Longitudinal Time
Code (LTC) used for synchroniZation. The combination of
48 kHZ (and 96 kHZ) “Word clocks”, together With the 24 fps
LTC, permits synchroniZation of commercially available
digital audio tape and disk systems With the present display
and input system over arbitrarily long periods of time. In
addition, using this inventive method of generating a Word
clock (either 48 or 96 kHZ), together With an LTC, alloWs the
audio to be started exactly When the video picture begins
playing or recording, thereby providing lip sync and other
sound-and-picture synchroniZation.
[0125] In some embodiments of the present invention, the
Well-knoWn digital or vertical interval time code (VITC) is
used for synchroniZation instead of using LTC for this
purpose. Also, some recently developed digital video sys
tems provide meta-data that carries such synchroniZation
information. HoWever, at present, the number of commer
cially available devices that synchroniZe based upon meta
data is small. Those skilled in the audio and video processing
arts shall appreciate that the present invention contemplates
compatibility With any of these synchroniZation formats and
techniques.
[0126] The three displayed frames associated With each 24
fps timecode can be distinguished in a number of Ways. For
example, timecode userbits can be used to carry these three
phase values. Also, metadata and other forms of user data
can carry this information. In systems Where metadata is
generaliZed, 72 fps and 75 fps data can be directly indicated,
23.98 rate for images. The present invention contemplates
embodiments that retain the 23.98 fps rate for images. As
described above, in one embodiment, the present invention
provides compatibility With the 23.98 fps rate images using
a frame rate clock having a frequency of 72* 1000/ 1001, or
71.928072 HZ. In one embodiment, Wherein a digital inter
face conforming to the incorporated SMPTE 292M digital
interface standard (or similar digital interface standard) is
used, this is accomplished by multiplying the 74.25 MHZ
pixel clock by the 1000/ 1001 reduction factor. This produces
a resultant pixel clock of 74.175842176 MHZ. This tech
nique is especially useful in eliminating a need for 48 kHZ
and 96 kHZ audio re-sampling (for example, When 48 kHZ
is tied to the 23.98 fps ?lm rate).
[0130] Pixel Replicate Zoom
[0131] In one embodiment of the present invention, a
magni?cation or “Zoom” method is available during output
(or “playback”) of the digital video stored in the buffer
memory 122. More speci?cally, in one embodiment, the
inventive system 100 includes a “pixel-replicate Zoom”
function that provides simple magni?cation of video images
during playback. As is Well knoWn, pixel-replicate Zoom
functions may be implemented by repeating pixel values that
are stored in a frame buffer for a selected number of
horiZontal magni?cation pixels (thereby producing a desired
horiZontal magni?cation), and restarting at the same scan
line starting address for a selected number of vertical
magni?cation pixels (thereby producing a desired vertical
magni?cation). Because it alloWs close scrutiny of moving
thus facilitating development of future systems synchro
niZed at 72 fps and 75 fps solely using metadata.
images, the pixel replicate Zoom function has proven useful
[0127] In one embodiment of the inventive display and
input system, audio data that is associated With each updated
[0132] Implementation
frame is stored With its associated frame in the FIFO
structured buffer memory 122. For example, if 24 fps images
are displayed by the system 100 at 72 HZ, 1/24 seconds of
associated audio is stored in the buffer memory 122 adjacent
to each associated video frame. Similarly, if 36 fps images
are displayed by the system 100 at 72 HZ, 1/36 seconds of
associated audio is stored in the buffer memory 122 adjacent
to each associated video frame.
[0128] As is Well knoWn, it is common practice in audio
systems to synchroniZe With the 1000/1001 ratio reduction
of 60 HZ video (i.e., 59.94 HZ) for compatibility With legacy
NTSC video. To achieve this synchroniZation, the audio is
sometimes locked to 48,000 digital samples corresponding
to 59.94 frames, and at other times corresponding to 60
frames. This necessitates audio re-sampling conversion in
some cases to adjust for this disparity. Other knoWn tech
niques, knoWn as “drop frame”, are utiliZed to adjust timing
and timecode marking at the end of 1000 (or 1001) frames
When utiliZing 59.94 HZ or When converting betWeen 59.94
HZ and 60 HZ.
for optimiZing quality and re?ning pixel processing.
[0133] The inventive display and input system may be
implemented in hardWare or softWare, or a combination of
both (e.g., programmable logic arrays). Unless otherWise
speci?ed, the algorithms included as part of the invention are
not inherently related to any particular computer or other
apparatus. In particular, various general purpose machines
may be used With programs Written in accordance With the
teachings herein, or it may be more convenient to construct
more specialiZed apparatus (e.g., integrated circuits) to per
form particular functions described above.
[0134] In one embodiment, the inventive display and input
system is implemented by modifying existing high resolu
tion display and input systems. For example, in one embodi
ment, the present invention is implemented by modifying
the commercially available “HDStationPRO” family of
products (including the commercially available “HDStation
PRO OEM Board”, models “HSO” (single-link, YUV-422)
and “HSO-DL” (dual-link, RGB-444) available from DVS
Digital Video, Inc. (hereafter “DVS”), having U.S. head
quarters in Glendale, Calif. Information regarding the func
tions performed by the HDStationPRO product family and
[0129] As described above, 24 fps ?lm is sometimes
adjusted by the 1000/1001 reduction factor for video output
HDStationPRO OEM Board, and speci?cations related
thereto, may be obtained by accessing the DVS Website
onto 59.94 HZ NTSC displays. As described above in more
located on the Internet
at “dvsus.com”. The infor
Feb. 27, 2003
US 2003/0038807 A1
mation published at the DVS Website relating to the HDSta
tionPRO products and HDStationPRO OEM Board is incor
system 100‘. As described above With reference to FIG. 1,
porated herein by reference.
[0135] As described in the incorporated related provi
sional application and in the incorporated DVS publications,
block 131 coordinate communications betWeen the com
the HDStationPRO OEM Board comprises a single-slot
display and input board for real-time input and output of
uncompressed HDTV images. As described in the related
provisional application, the HDStationPRO OEM Board
display and input board (including daughter board) inter
faces to a computer using a 64-bit PCI-Bus interface. The
display and input board provides synchroniZed digital video
and audio input, synchroniZed digital and analog video
output, and digital audio output at common television for
mats using 50 HZ and 60 HZ interlaced and non-interlaced
the control/status registers and interrupt control processing
puter and the display and input system 100‘. More speci?
cally, as described above With reference to FIG. 1, in
accordance With one aspect of the present invention, the
computer Waits until the inventive display and input system
100‘ signals the computer that the display of a needed video
frame is complete, and that an associated frame buffer is
available for use by the computer. As described above, this
synchroniZation betWeen the computer and the inventive
system 100‘ is achieved using either an interrupt signal (such
as a synchroniZation ?ag) or using a control/status register
accessible to the computer. In the exemplary implementation
100‘ of FIG. 2, this synchroniZation is achieved using the
display.
block 131.
[0136] As described in the incorporated DVS references
and related provisional application, the HDStationPRO
OEM Board includes ?eld programmable circuits (e. g., ?eld
[0140] The exemplary implementation shoWn in FIG. 2
also includes input and output embedded audio (also
programmable gate array (FPGA) circuits). The ?eld pro
grammable circuits can be programmed to adjust some of
the functions performed by the video processing circuitry. In
one exemplary embodiment, the present invention is imple
mented by programming the HDStationPRO OEM Board
and thereby modifying existing clock signals (and/or pro
viding additional clock signals) to include clock frequencies
referred to as “Audio in Video”) signal paths. As shoWn in
FIG. 2, an embedded input audio signal path (“AiVin”) 210
is coupled betWeen the audio/video de-serialiZer 104 and the
digital audio I/O controller 119. Similarly, an embedded
output audio signal path (“AiVout”) 212 carries embedded
output audio from the audio I/O controller 119 for input to
the audio/video serialiZer 114.
[0141] As described above, the HDStationPRO OEM
and formats necessary for performing the inventive func
tions described above.
Board includes programmable circuits that can be pro
[0137]
video processing circuitry. In the exemplary implementation
FIG. 2 shoWs a block diagram of one exemplary
implementation 100‘ of the display and input system of FIG.
1 using the HDStationPRO OEM Board. Many of the blocks
shoWn in the exemplary implementation of FIG. 2 perform
similar functions to those described above With reference to
FIG. 1, and therefore are not described in more detail herein.
[0138] Similar to the display and input system described
above With reference to FIG. 1, the exemplary implemen
grammed to adjust some of the functions performed by the
100‘ shoWn in FIG. 2, the clock synchroniZation system 126
includes a micro-programmable video clock and raster
generator block having PLL circuitry. As shoWn in FIG. 2,
the system 100‘ includes a control line “TCload”214 coupled
from the control/status registers and interrupt control pro
cessing block 131 to the micro-programmable video clock
and raster-generator 126. The control line TCload 214 is
used to load the micro-programmable clock generator With
tation 100‘ shoWn in FIG. 2 includes the folloWing video
interfaces: an optional video input block including a digital
control code. As described above, the control code is used to
video input interface 104 (audio/video de-serialiZer) and
optional analog-to-digital
converter 106, an optional
clock signals) to include clock frequencies and formats
necessary for performing the various functions of the present
modify existing clock signals (and/or provide additional
analog video output block including a video RAMDAC 110,
and an optional digital video output block including a digital
invention.
video output interface 114 (audio/video serialiZer). The
[0142] The exemplary implementation shoWn in FIG. 2
exemplary embodiment 100‘ also includes an optional audio
also includes tWo signals, TCout 216 and TCin 218, Which
are used to provide programmable timing control of the
video input 104 and output blocks 110, 114. As shoWn in
input/output 116. The optional audio input/output 116
includes a digital audio I/O controller 119. The exemplary
implementation of the synchroniZed display and input sys
FIG. 2, TCout 216 is output by the micro-programmable
tem 100‘ also includes a buffer memory 122, color space
transform or converter blocks 124, and clock synchroniZa
video clock and raster generator clock synchroniZation block
126 and input to both the audio/video serialiZer 114 and the
video RAMDAC 110. Similarly, TCin 218 is output by he
micro-programmable video clock and raster generator clock
synchroniZation block 126 and input to the audio/video
de-serialiZer 104.
tion blocks 126 including clock synchroniZation circuitry.
The implementation shoWn in FIG. 2 also includes a com
puter interface 128. The computer interface permits access
to the buffer memory 122 by a computer (not shoWn)
connected to the Well knoWn PCI bus 200. The implemen
tation shoWn in FIG. 2 also includes a data bus sWitch 202,
a dual FIFO buffer, a FIFO buffer 206, and a video bus
sWitch 208.
[0139] As shoWn in FIG. 2, the computer interface 128
also includes a DMA engine 129 and control/status registers
and interrupt control processing 131. The DMA engine 129
functions in a Well knoWn manner to alloW direct memory
access betWeen the computer and the display and input
[0143] As shoWn in FIG. 2, the implementation 100‘ also
uses tWo timing signals, “WC”220 and “FS”222 to synchro
niZe the audio I/O to the video timebase. Speci?cally, the
WC 220 timing signal provides a “Wordclock” signal to the
digital audio I/O controller 119. The PS 222 timing signal
provides a “framestart” signal to the digital audio I/O
controller 119. These timing signals function similarly to the
timing signals described above With reference to the display
and input system 100 of FIG. 1. The implementation 100‘ of
Feb. 27, 2003
US 2003/0038807 A1
timebase signals (“H, V, He and Ve”) used by the video clock
synchronization system. These signals function similarly to
tionPRO product family (and more speci?cally, the DVS
HDStationPRO OEM Board), it Will be understood by those
skilled in the video processing and display arts that this
the analogous signals described above With reference to the
system of FIG. 1.
present invention can be implemented in hardWare, soft
FIG. 2 also includes internal horizontal and vertical video
[0144]
The invention may also be implemented in one or
more computer programs executing on one or more pro
grammable computer systems each comprising at least one
processor, at least one data storage system (including vola
tile and non-volatile memory and/or storage elements), at
least one input device or port, and at least one output device
or port. Program code is applied to input data to perform the
functions described herein and generate output information.
The output information is applied to one or more output
implementation is exemplary only. As described above, the
Ware, or a combination of hardWare and softWare. Moreover,
the present invention may be implemented by computer
programs executed by special-purpose or general purpose
computing devices, or both. Therefore, the scope of the
present inventive display and input system is not limited to
any of the exemplary implementations described above.
[0148] Accordingly, it is to be understood that the inven
tion is not to be limited by the speci?c illustrated embodi
ments, but only by the scope of the appended claims.
devices, in knoWn fashion.
[0145]
Each such program may be implemented in any
desired computer language (including machine, assembly, or
high level procedural, logical, or object oriented program
ming languages) to communicate With a computer system.
In any case, the language may be a compiled or interpreted
What is claimed is:
1. A fully synchroniZed audio/video display and input
system capable of displaying fully synchroniZed video on
computer-compatible display monitors, comprising:
a) an optional video input/output (I/O) having a ?rst
language. Each such computer program is preferably stored
plurality of associated video clock signals for commu
on or doWnloaded to a storage media or device (e.g., solid
nicating video information;
state memory or media, or magnetic or optical media)
readable by a general or special purpose programmable
computer, for con?guring and operating the computer When
the storage media or device is read by the computer system
to perform the procedures described herein. The inventive
system may also be considered to be implemented as a
computer-readable storage medium, con?gured With a com
puter program, Where the storage medium so con?gured
b) an optional audio I/O having a ?rst plurality of asso
ciated audio clock signals for communicating audio
information;
c) a frame buffer memory, coupled to the optional audio
I/O, Wherein video frames are stored Within and
retrieved from the frame buffer memory on a ?rst in,
?rst out (FIFO) basis;
causes a computer system to operate in a speci?c and
prede?ned manner to perform the functions described
herein.
d) a computer interface, adapted to interface to a computer
and coupled to the frame buffer memory; and
[0146] A number of embodiments of the present invention
have been described. Nevertheless, it Will be understood that
various modi?cations may be made Without departing from
the spirit and scope of the claimed invention. For example,
although some embodiments of the present inventive display
and input system have been described above as using digital
interfaces that conform to the SMPTE 292M digital inter
face standard, it Will be understood that the scope of the
present invention is not limited to use of this speci?c digital
standard. Rather, the scope of the present invention encom
passes any useful or convenient digital computer display
interface. For example, the present invention is compatible
With DVI digital. Similarly, although some embodiments
have been described as using speci?c analog video inter
faces, it Will be understood that the scope of the present
e) a clock synchroniZation system adapted to receive the
?rst plurality of video and audio clock signals, Wherein
the clock synchroniZation system generates a second
plurality of video and audio clock signals used inter
nally by the display and input system to store and
invention is not limited to use With the analog video formats
given in the examples above. Rather, the present inventive
display and input system can be used With any useful or
convenient analog video format. For example, the present
invention is compatible With RGB, YCrCb and YC (also
knoWn as S-video).
[0147] Further, although the description of the exemplary
embodiments provided above uses exemplary digital image
formats (such as, e.g., the image formats de?ned in the
SMPTE 296M standard), it Will be understood that the
present inventive display and input system can accommo
date any useful or convenient image format. Finally,
although one described implementation of the present inven
tion makes use of the commercially available DVS HDSta
retrieve audio and video information Within and from
the frame buffer memory;
Wherein the display and input system generates fully
synchroniZed audio and video at selected frame rates,
and Wherein the selected frame rates exceed 60 HZ.
2. The display and input system of claim 1, Wherein video
frames are accessed from the frame buffer memory on a
frame by frame basis, and Wherein a selected video frame is
automatically displayed a pre-determined repeat number of
times at a selected output video frame rate.
3. The display and input system of claim 2, Wherein
selected video frames are repeated three times to synchro
niZe 24 fps video for display on a 72 HZ monitor.
4. The display and input system of claim 2, Wherein
selected video frames are tWice repeated to synchroniZe 36
fps video for display on a 72 HZ monitor.
5. The display and input system of claim 2, Wherein
selected video frames are repeated three times to synchro
niZe 25 fps video for display on a 75 HZ monitor.
6. The display and input system of claim 2, Wherein
selected video frames are tWice repeated to synchroniZe 37.5
fps video for display on a 75 HZ monitor.
7. The system of claim 2, Wherein the selected output
video frame rate is reduced by a refresh rate reduction factor
Feb. 27, 2003
US 2003/0038807 A1
of 1000/1001 thereby providing compatibility With legacy
12 fps video With a 72 HZ monitor.
25. The system of claim 22, Wherein the video data rate
comprises 72 fps, and Wherein the 72 fps video data rate is
divided by a factor of 3 thereby producing a 24 fps sub-rate,
and Wherein the LTC is synchroniZed to every third frame of
the 72 fps video.
9. The system of claim 2, Wherein the selected video
frames are repeated siX times providing synchroniZation of
26. The system of claim 25, Wherein video frames asso
ciated With each time code are distinguished from each other
12.5 fps video With a 75 HZ monitor.
10. The system of claim 2, Wherein the selected video
using userbits carrying phase information associated With
NTSC monitors and other 59.94 HZ video systems.
8. The system of claim 2, Wherein the selected video
frames are repeated siX times providing synchronization of
frames are repeated ?ve times providing synchroniZation of
12 fps video With a 60 HZ monitor.
each video frame.
27. The system of claim 25, Wherein video frames asso
ciated With each time code are distinguished from each other
11. The system of claim 1, Wherein the display and input
system synchroniZes the ?rst plurality of video and audio
clock signals to the second plurality of video and audio clock
using metadata carrying phase information associated With
signals.
output by the display and input system through the optional
video I/O comprises analog video.
12. The system of claim 11, Wherein the system locks the
?rst and second plurality of video and audio clock signals to
one another using phase-locked loop clock synchroniZation.
each video frame.
28. The system of claim 1, Wherein the video information
29. The system of claim 28, Wherein the analog video
comprises RGB video.
13. The system of claim 11, Wherein the optional video
30. The system of claim 1, Wherein the video information
I/O inputs a video data rate for use by the system, and
output by the display and input system through the optional
video I/O comprises digital video.
Wherein the second plurality of internally generated clock
signals are derived from and synchroniZed With the input
31. The system of claim 30, Wherein the digital video
video data rate.
conforms to an SMPTE 292M standard for HDTV bit-serial
14. The system of claim 13, Wherein the optional video
I/O inputs a digital video signal having an associated data
rate clock, and Wherein the second plurality of internally
generated clock signals are derived from and synchroniZed
digital interfaces.
With the data rate clock.
memory holds at least tWo video frames.
15. The system of claim 13, Wherein the optional video
I/O inputs an analog video signal having associated hori
Zontal and vertical synchroniZation signals, and Wherein the
second plurality of internally generated clock signals are
derived from and synchroniZed With the horiZontal and
vertical synchroniZation signals.
16. The system of claim 15, Wherein the second plurality
of internally generated clock signals are derived from and
synchroniZed With the horiZontal and vertical synchroniZa
tion signals using a phase-locked loop (PLL) clock genera
tion circuit.
17. The system of claim 11, Wherein audio U0 is syn
chroniZed to video I/O using a Word clock.
18. The system of claim 17, Wherein the display and input
system generates an audio sample clock for use by an audio
I/O device, Wherein the audio sample clock is derived from
the second plurality of video clock signals.
19. The system of claim 18, Wherein the audio sample
clock comprises a 48 kHZ audio sample clock.
20. The system of claim 18, Wherein the audio sample
clock comprises a 96 kHZ audio sample clock.
21. The system of claim 11, Wherein audio information is
stored Within the frame buffer memory at a location that is
logically related to its associated and respective video infor
mation.
22. The system of claim 17, Wherein the audio U0 is
synchroniZed to the video I/O using Longitudinal Time
Codes (LTC) associated With a frame sub-rate derived from
the video I/O, and Wherein the LTC is used for synchroni
Zation of the audio With digital audio devices.
23. The system of claim 17, Wherein the audio U0 is
synchroniZed to the video I/O using Vertical Interval Time
Codes (VITC).
24. The system of claim 11, Wherein the audio U0 is
synchroniZed to the video I/O using metadata.
32. The system of claim 30, Wherein the digital video
comprises Digital Video Interactive (DVI) digital video.
33. The system of claim 2, Wherein the frame buffer
34. The system of claim 33, Wherein the frame buffer
memory compensates for timing variations betWeen the
computer and the display and input system.
35. The system of claim 33, Wherein the frame buffer
memory compensates for timing variations in preparation of
video displays.
36. The system of claim 2, further comprising an optional
color space transformation (CST) block operatively coupled
to the optional video I/O.
37. The system of claim 1, Wherein the display and input
system selectively outputs the fully synchroniZed audio and
video for display on 72 HZ and 75 HZ computer-compatible
monitors.
38. The system of claim 1, Wherein the display and input
system selectively outputs the fully synchroniZed audio and
video to a digital video interface.
39. The system of claim 1, Wherein the display and input
system selectively outputs the fully synchroniZed audio for
input to an eXternal analog audio device.
40. The system of claim 1, Wherein the display and input
system selectively outputs the fully synchroniZed audio for
input to an eXternal digital audio device.
41. The system of claim 31, Wherein relatively high
precision piXel values are used to represent the digital video
information.
42. The system of claim 41, Wherein at least 8-bits are
used to represent each color component of the digital video.
43. The system of claim 41, Wherein 10-bits are used to
represent each color component of the digital video.
44. The system of claim 41, Wherein at least 12-bits are
used to represent each color component of the digital video.
45. A fully synchroniZed audio/video display and input
system capable of displaying fully synchroniZed video on
computer-compatible display monitors, comprising:
Feb. 27, 2003
US 2003/0038807 A1
a) means for optionally inputting and outputting (I/O)
52. The method of fully synchroniZing input and display
video information having a ?rst plurality of associated
of audio and video information for use With computer
video clock signals;
compatible display monitors of claim 51, further compris
b) means for optionally inputting and outputting (I/O)
audio having a ?rst plurality of associated audio clock
ing:
a) optionally performing color space transformations
signals;
c) means, responsive to the optional audio and video I/O
means, for storing video frames, Wherein video frames
are stored Within and retrieved from the storage means
on a ?rst in, ?rst out (FIFO) basis;
d) means, coupled to the storage means, for interfacing
With a computer; and
e) means, responsive to the ?rst plurality of video and
(CST) on video information from a ?rst color space to
a second color space.
53. The method of fully synchroniZing input and display
of audio and video information for use With computer
compatible display monitors of claim 51, further compris
mg:
a) selectively displaying the fully synchroniZed audio and
video generated in step e) on 72 HZ and 75 HZ
computer-compatible monitors.
audio clock signals, for performing clock synchroniZa
54. The method of claim 51, further comprising:
tion, Wherein the clock synchroniZation means gener
ates a second plurality of video and audio clock signals
a) selectively outputting the fully synchroniZed audio and
used internally by the display and input system to store
video to a digital video interface.
and retrieve audio and video information Within and
from the storage means;
55. The method of claim 51, further comprising:
Wherein the display and input system generates fully
input to an external analog audio device.
56. The method of claim 51, further comprising:
synchroniZed audio and video at selected frame rates,
and Wherein the selected frame rates eXceed 60 HZ.
46. The fully synchroniZed audio/video display and input
system of claim 45, further comprising:
a) means, responsive to the optional video I/O means and
coupled to the storage means, for performing color
space transformations (CST) from a ?rst color space to
a second color space.
47. The system of claim 45 , Wherein the display and input
system selectively outputs the fully synchroniZed audio and
video for display on 72 HZ and 75 HZ computer-compatible
monitors.
48. The system of claim 45 , Wherein the display and input
system selectively outputs the fully synchroniZed audio and
video to a digital video interface.
49. The system of claim 45, Wherein the display and input
system selectively outputs the fully synchroniZed audio for
input to an eXternal analog audio device.
50. The system of claim 45 , Wherein the display and input
system selectively outputs the fully synchroniZed audio for
a) selectively outputting the fully synchroniZed audio for
a) selectively outputting the fully synchroniZed audio for
input to an external digital audio device.
57. Afully synchroniZed audio/video input system for use
With computers, comprising:
a) an optional video input having a ?rst plurality of
associated video input clock signals for communicating
video information;
b) an optional audio input having a ?rst plurality of
associated audio input clock signals for communicating
audio information;
c) a frame buffer memory, coupled to the optional audio
input and capable of storing video frames received
from the optional video input, Wherein video frames are
stored Within and retrieved from the frame buffer
memory on a ?rst in, ?rst out (FIFO) basis;
d) a computer interface, adapted to interface to a computer
and coupled to the frame buffer memory; and
input to an eXternal digital audio device.
51. A method of fully synchroniZing input and display of
audio and video information for use With computer-compat
ible display monitors, comprising:
a) optionally inputting and outputting (I/O) video infor
mation having a ?rst plurality of associated video clock
signals;
b) optionally inputting and outputting (I/O) audio having
a ?rst plurality of associated audio clock signals;
c) generating a second plurality of video and audio clock
signals, Wherein the ?rst and second plurality of video
and audio clock signals are locked to one another;
d) accessing video frames in a storage means on a ?rst in,
?rst out (FIFO) basis; and
e) a clock synchroniZation system adapted to receive the
?rst plurality of video and audio input clock signals,
Wherein the clock synchroniZation system generates a
second plurality of video and audio clock signals used
internally by the input system to store audio and video
information Within the frame buffer memory;
Wherein the input system generates fully synchroniZed
audio and video at selected video frame rates, and
Wherein the selected video frame rates eXceed 60 HZ.
58. The fully synchroniZed audio/video input system of
claim 57, Wherein the selected video frame rates comprise
72 fps and 75 fps video frame rates.
59. The fully synchroniZed audio/video input system of
claim 57, further comprising:
e) generating fully synchroniZed audio and video at
a) a color space transformation (CST) block operatively
coupled to the optional video input, Wherein the CST
selected frame rates, Wherein the selected frame rates
eXceed 60 HZ.
block optionally converts video from a ?rst color space
representation to a second color space representation.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement