S`M?i?` 6
US 2007014653 8A1
(19) United States
(12) Patent Application Publication (10) Pub. No.: US 2007/0146538 A1
(43) Pub. Date:
Kakinuma et al.
(54)
IMAGE PICKUP APPARATUS
Publication Classi?cation
(75) Inventors: Minoru Kakinuma, Hachioji-shi (JP);
Hidetoshi Fukuda, Yokohama-shi (JP);
(51) Int. Cl.
H04N 5/235
Hiroaki Koseki, Kokubunji-shi (JP);
(52)
Takumi Momose, Hachioji-shi (JP);
Osamu Inagaki, Hachioji-shi (JP)
(57)
Correspondence Address:
(73)
21
)
(22)
(2006.01)
US. Cl. ............................................................ .. 348/362
ABSTRACT
.
.
.
WESTERMAN, HATTORL DANIELS &
short-time exposure image data and a long-time exposure
ADRIAN, LLP
1250 CONNECTICUT AVENUE, NW
SUITE 700
WASHINGTON, DC 20036 (Us)
'
Asslgnee? OLYMPUS OPTICAL CO‘:
image data of the same object to be photographed are stored
IO SE memory and LE memory and then transferred IO CPU,
motion thereof being detected at a motion detecting section
on the basis of the tWo image data; if no motion is detected,
a Wide dynamic range, synthesized image as synthesized at
LTD:
Tokyo (JP)
(
Jun. 28, 2007
A
pp
1. N .1
a synthesizing circuit on the basis of image data read out
from the tWo memories and compressed is outputted through
lector controlled b y CPU. If a motion is detected ’ the
long-time exposure image data read out from the LE
11/706 291
a Se
Feb_ 15, 2007
memory and processed at a signal processing circuit is
Related US Application Data
having a function for generating Wide dynamic range, syn
0
Filed;
’
outputted through the selector. An image pickup apparatus
_ _ _
_
_
thesized image by synthesizing image signals corresponding
(62) 2135/1155; of apphcanon NO‘ 09/358’657’ ?led on Jul‘
’
(30)
'
to a plurality of frames of different exposure amounts is
thereby able to perform ON/OFF sWitching of the generating
Foreign Application Priority Data
function of synthesized image on the basis of previously
taken image data, so that a failed synthesized image can be
Jul. 28, 1998
(JP) ......................................... .. 10-226575
prevented from being outputted.
5
S’M?i?‘
MD
PROCESSING _\
Cl RCU | T
6
9
7 151151115
5
CIRCUIT
JPEG
[l O
8
‘A
, 8A
:r
CONTROLLER
'
CPU
§
STROBE
'
DISPLAY
LCD
I14
MEMORY CARD
l /F
22
r13
'
CIRCUIT
g
DRAM
r12
i
21
[l 1
MEMORY
I15
MEMORY CARD
r16
1
17
PC
/F
INPUT KEY
I
P
"""" ’ "
18
[
RwEER
fl 9
------- -»>PRINTER
Patent Application Publication Jun. 28, 2007 Sheet 1 0f 8
OZ
1
_
NZm?
E25‘2%
9
wzGmoaC290
m
8%
,$3258
w?
1
a
v?w?
I
m
B
I
2Q3,.520 E90250:$26952 oE.In, ? $5 N5A73:
:SEQ
m:
-“E1.
8
‘
mm
mM
M
A
00D
US 2007/0146538 A1
2
W
m oEw
M
:m2Ez_
Patent Application Publication Jun. 28, 2007 Sheet 3 0f 8
AE, AF BEFORE
IMAGE TAKING
F I G. 5A
I
US 2007/0146538 A1
81
I
TAKE SL IMAGES ~32
DETERMINATION
BASED ON TWO IMAGES
WHETHER SL SYNTHESIS
SUITABLE OR NOT
UNSUITABLE
SUITABLE
,
OUTPUT 3L
S25
OUTPUT LE
IMAGE
SYNTHESIZED
|MAGE
F I G. 58
SI 1
DETECT
MOTION FROM TWO
IMAGES
MOTION DETECTED
N0 MOTION DETECTED S212
sgs
I
OUTPUT 5L
OUTPUT LE
SYMTIJEEEZED
IMAGE
F I G. 5C
51 I
DETECT
MOTION
NO MOTION DETECTED
OUTPUT SL
MOTION DETECTED
S14
LESS THAN
ALLOWABLE AMOUNT FOR
MOTION AMOUNT
SYNTHESIZED
IMAGE
OUTPUT 5L
SYNTHESIZED
IMAGE
OUTPUT LE
IMAGE
Patent Application Publication Jun. 28, 2007 Sheet 4 0f 8
US 2007/0146538 Al
F | 6.6
*5
is"
E CAMERA SIGNAL
_’ A/D
PROCESSING CKT‘+
A
V
AE,AWB
7’~DETECTION
CIRCUIT
21
V S8 V
S
STROBE
‘
>
CPU
AF CIRCUIT
23S
If ‘88
F I G. 7
5
__
‘
5
_
_
/6-1
_
_
6-3
_‘,5
6—4/6_5
+ A/D . ‘SE MEMORY »<>I>+
I
_
’
SYNTHES|-_>C(S)%|F|’5E%
zING CKT
N + I_
‘LE IMAGE
\
CKT
a _,
_.l
_
LLI
L SIGNAL /6 6
w
PROCESSING
S
CKT
'
N8
85
CPU
L _ _ _ _J
,
_
'
—_6\—7
Patent Application Publication Jun. 28, 2007 Sheet 5 0f 8
US 2007/0146538 Al
F | G. 8
+
+
+
+
+
F I G. 9
5
6'
5
$
\ CAMERA SIGNAL
_’ A/D
PROCESSING cm“
I
f
AF,AWB
7”~DETECTION
CIRCUIT
21
g
STROBE
8
I
\
CPU
AE CIRCUIT
5
243
‘80
F I G. 1 O
Patent Application Publication Jun. 28, 2007 Sheet 6 0f 8
US 2007/0146538 A1
5
6
3.
§
__, M)
.
F|G.11
C CAMERA SIGNAL _,
PRCCESSINC CKT
I
_
V
AE,AE,AwB
7~DETECTION
CKT
21
g3
S
STROBE
f
‘
\
CAMERA SHAKE / 25
DETECTION
mom
CPU
6
5
S
I
S’Né?ii’?
*’ A/D
.F|G.12
‘ PROCESSING *
26
3 f
MOTION
iI
AE, AF, AWB
7 “4 DETEOT | ON DETECTION
CKT
O l RCU I T
m
21% STROBE
‘
CPU
CKT
‘I
Patent Application Publication Jun. 28, 2007 Sheet 7 0f 8
F I G. I 3
A
\
mAnD>6Em<A5Zv
F 85.42
86
1/ 1/ 1/1/1/‘/ 1/
15 30 60 125 250
500 1000
SHUTTER SPEED (TV)
F I G. I 4
SE50
LIGHT AMOUNT
F I G. I 5
HEADER ~41
SHORT-TIME
EXPOSURE ~42
IMAGE DATA
LONG-T I ME
EXPOSURE ~43
IMAGE DATA
US 2007/0146538 Al
Patent Application Publication Jun. 28, 2007 Sheet 8 0f 8
US 2007/0146538 A1
FIG. I6
I
START
I
f
REFER TO HEADER
TO DISCRIMINATE ~S2I
EXPOSURE AMOUNT
OOMPUTE EXPOSURE
AMOUNT RATIO AND
~S22
DETECT UNSUITABLE
PORTION FOR SYNTHESIS
S23
SYNTHESIZING
IS POSSIBLE
YES
8224
SELECT LONG-TIME
S26
REFERRED
UNSUITABLE PORTION
I
EXPOSURE IMAGE
PIXEL IS IN
FOR SYNTHESIS
'
327
S228
CORRECTION OF UN
N0
SUITABLE PORTION
FOR SYNTHESIS
SYNTHESIZING
>
,
S25
I
GRADATION
‘
'
DYNAMIC RANGE
COMPRESSION ~33‘)
END
OHARACTERISTIC
CONVERSION
US 2007/0146538 A1
IMAGE PICKUP APPARATUS
Jun. 28, 2007
[0006]
Further, a disclosure has been made in Japanese
RELATED APPLICATIONS
patent application laid open No. 5-64075 With respect to a
video camera including: image pickup means including a
[0001] This application is a divisional of application U.S.
patent application Ser. No. 09/358,657, ?led on Jul. 26,
1999, Which claims the bene?t of Japanese Application No.
10-226575 ?led in Japan on Jul. 28, 1998, the disclosures of
Which are incorporated herein by reference in their entirety.
nals for representing an object image; motion detection
means for determining based on the image signals obtained
from the image pickup means Whether a motion exceeding
solid-state image pickup device, for outputting image sig
a predetermined amount occurs in the object or not; means
BACKGROUND OF THE INVENTION
for controlling exposure amount so that the image pickup
means takes images of the object at tWo different exposure
[0002] The present invention relates to image pickup
apparatus in Which image signals corresponding to a plu
the motion detection means; and means for forming synthe
rality of frames at different exposure amounts are outputted
from an image pickup device and are synthesiZed to obtain
an image having Wide dynamic range, and also relates to
representing an image at the larger exposure amount, those
apparatus related thereto.
nance in that image With those image signals representing
[0003]
Solid-state image pickup devices such as CCD
amounts on condition that a motion has not been detected by
siZed image signals by replacing, of the image signals
image signals representing regions of relatively high lumi
the corresponding regions in an image at the smaller expo
sure amount Also, the same publication discloses a video
image pickup device are generally used in image pickup
camera Which furthermore includes high luminance region
apparatus, as TV camera, video camera, electronic camera,
detection means for determining Whether or not, of the
etc. There is a problem hoWever that the dynamic range of
a solid-state image pickup device is much narroWer than that
of a silver salt photographic ?lm.
[0004]
To eliminate this problem, there have been pro
posed techniques in Which image signals corresponding to
tWo frames at different exposure amounts are read out from
a single image pickup device and are synthesiZed to obtain
an image having an increased dynamic range. For example,
an image pickup apparatus having the folloWing construc
tion is disclosed in Japanese Patent Publication No.
2522015. In particular, a disclosure has been made With
respect to an image pickup apparatus including: image
pickup means for converting object image into electrical
signals; image pickup control means for selectively execut
ing a ?rst mode or second mode, said ?rst mode for
successively outputting images at different exposure
amounts in cycles by cyclically changing accumulating time
at the image pickup means, image portions at suitable level
being synthesiZed to form a synthesiZed frame of said
predetermined cycle by respectively comparing signal level
of each portion of a plurality of frames at different exposure
amounts outputted from said image pickup means With a
predetermined reference value, said second mode for pro
viding output by using a constant accumulating time at said
image pickup means; and sWitch control means for sWitch
ing the image pickup control means to the ?rst mode When
an existence of object having a large difference in luminance
Within a frame is detected in said second mode by comparing
signals obtained from said image pickup means With a
above described image signals for representing an image at
the larger exposure amount, the regions of relatively high
luminance in that image exceed a predetermined area. The
above described exposure amount control means controls
the exposure amount so that images can be picked up at tWo
different exposure amounts on condition that no motion has
been detected by the above motion detection means and that
a high luminance region has been detected by the above
described high luminance region detection means.
[0007] In the above described image pickup apparatus as
disclosed in Japanese Patent Publication No.2522015, hoW
ever, the mode sWitching means for sWitching to the ?rst
mode for generating image having a Wide dynamic range is
to sWitch the mode When an existence of object having a
large difference in luminance is detected Within a frame
obtained in the second mode. The disclosure merely refers to
the sWitching of mode depending on the conditions only of
luminance values and other conditions concerning the gen
eration of Wide dynamic range image Were not at all taken
into consideration.
[0008] Further, in Japanese patent application laid open
No.5-64075, motion of the object is detected from image
signals before taking of image and a synthesiZed image is
obtained by picking up images at tWo different exposure
amounts on condition that motion of the object is absent.
Thus the detecting of motion of the object from something
other than the image signals before image taking Was not
taken into consideration. Other conditions for generating a
predetermined signal level. This image pickup apparatus,
synthesiZed image except the detection of motion of the
When in the ?rst mode, is capable of obtaining an image of
object Were not taken into consideration either.
Which all the portions of the frame are at suitable signal
level. Even When an object having a large difference in
SUMMARY OF THE INVENTION
luminance exists Within a frame as in a backlighted condi
tion, lack of detail at high level portion for example does not
occur Within the frame and it is in effect possible to Widen
dynamic range thereof.
[0005] Further, Without a determination made by the
operator on the conditions of the object, etc., sWitching to
the ?rst mode is automatically made by the sWitching
control means When an object having a large difference in
luminance exists Within the frame in the second mode,
making it possible to automatically correct a backlighted
condition, etc.
[0009] The present invention has been made to eliminate
the above described problems in the conventional image
pickup apparatus having a function for synthesiZing tWo
image signals of different exposure amounts to generate
Wide dynamic range, synthesiZed image. Its main object is to
provide an image pickup apparatus in Which ON/OFF
sWitching of processing for the generation of Wide dynamic
range, synthesiZed image can be performed at high accuracy.
[0010]
In a ?rst aspect of the invention, there is provided
an image pickup apparatus including: image pickup means
US 2007/0146538 A1
capable of taking images of the same object at a plurality of
different exposure amounts to generate image signals cor
responding to a plurality of frames at different exposure
amounts; and means for generating Wide dynamic range,
synthesized image by synthesizing image signals corre
sponding to a plurality of frames at different exposure
amounts obtained by the image pickup means. It further
includes means for determining Whether or not a suitable
synthesiZed image can be obtained on the basis of image
signals corresponding to a plurality of previously taken
frames; and means for controlling ON/OFF sWitching of the
generation processing operation at the means for generating
synthesiZed image based on an output of the determination
Jun. 28, 2007
thus provided so as to perform synthesiZing process With
correcting the amount of motion When the motion amount is
Within the alloWable range for correction, it is possible to
provide a grater range for the conditions of object of Which
a Wide dynamic range, synthesiZed image can be formed.
The above object is thereby accomplished.
[0018] It is yet another object of the invention to provide
an image pickup apparatus capable of taking images suitable
for the generation of Wide dynamic range, synthesiZed
image Without requiring detection of the conditions for
generating Wide dynamic range, synthesiZed image after the
taking of image.
[0019]
means.
[0011] Since ON/OFF sWitching control of the generation
processing operation at the means for generating Wide
dynamic range, synthesiZed image is thus performed based
on image signals corresponding to a plurality of previously
taken frames, an image pickup apparatus can be achieved as
capable of accurately performing ON/OFF sWitching control
of processing operation for the generation of Wide dynamic
range, synthesiZed image. The above main object is thereby
In a fourth aspect of the invention, there is provided
an image pickup apparatus including: image pickup means
capable of taking images of the same object at a plurality of
different exposure amounts to generate image signals cor
responding to a plurality of frames at different exposure
amounts; and means for generating Wide dynamic range,
synthesiZed image by synthesiZing image signals corre
sponding to a plurality of frames at different exposure
amounts obtained by the image pickup means. It further
accomplished.
includes means for determining Whether or not a suitable
[0012] It is another object of the present invention to
provide an image pickup apparatus in Which, When the
object has been moved, it is possible to prevent a failure of
tion of the object obtained before the taking of image; and
means for controlling ON/OFF sWitching of the generation
processing operation at the means for generating synthesiZed
synthesiZed image can be obtained on the basis of informa
Wide dynamic range, synthesiZed image.
image based on an output of the determination means.
[0013] In a second aspect of the invention, the determi
nation means of the image pickup apparatus according to the
?rst aspect includes means for detecting motion in the object
image and the generation processing operation at the means
for generating synthesiZed image is sWitched to OFF When
motion in the object image has been detected by the motion
[0020] ON/OFF switching of processing operation for the
detection means.
[0014] Since the motion detection means is thus provided
so as not to perform processing operation for the generation
of synthesiZed image if a motion in the object image has
been detected, it is possible to prevent a failure of Wide
dynamic range, synthesiZed image When the object has been
moved. The above object is thereby accomplished.
[0015] It is still another object of the invention to provide
an image pickup apparatus in Which the feasible range for
image taking can be made greater With respect to the
conditions of object of Which a Wide dynamic range, syn
thesiZed image can be formed.
[0016] In a third aspect of the invention, the determination
means of the image pickup apparatus according to the ?rst
aspect includes means for detecting motion in the object
image and motion amount determination means for deter
mining Whether or not the amount of motion detected by the
motion detection means is Within an alloWable range for
generation of Wide dynamic range, synthesiZed image can
thus be effected relying on object information obtained
before the taking of image such as the information on
detection of motion of the object Which is obtained based on
an output from an external AF detection circuit or external
AE detection circuit. An image pickup apparatus can be
achieved as capable of taking images suitable for the gen
eration of Wide dynamic range, synthesiZed image. The
above object is thereby accomplished.
[0021]
It is a further object of the invention to provide an
image pickup apparatus adapted to be capable of readily
generating Wide dynamic range, synthesiZed image even of
an object involving a motion oWing to the fact that the
motion of the object can in effect be stopped by strobe image
taking.
[0022] In a ?fth aspect of the invention, an image pickup
apparatus according to the fourth aspect includes strobe
emission means. In addition, the determination means
thereof includes means for detecting motion in the object
image so that, When a motion in the object image has been
detected by the motion detection means, images are taken in
a forced strobe emission mode to perform the generation
processing of synthesiZed image.
correction. The apparatus further includes motion correction
[0023]
means for, When the amount of motion is determined as
emission mode and the generation processing of synthesiZed
Within the alloWable range for correction by the motion
amount determination means, correcting the amount of
Since images are thus taken in a forced strobe
image is performed When a motion in the object has been
motion based on such determination output so that synthe
detected at the motion detection means, the motion in the
object can in effect be stopped even of an object involving
siZing process be performed With correcting the amount of
motion of the image signals corresponding to a plurality of
dynamic range, synthesiZed image. The above object is
frames.
thereby accomplished.
[0017]
Since the motion detection means and the motion
amount determination and the motion correction means are
a motion, making it possible to readily generate a Wide
[0024]
It is a further object of the invention to provide an
image pickup apparatus adapted to be capable of also readily
US 2007/0146538 A1
generating Wide dynamic range, synthesized image even of
Jun. 28, 2007
[0032] Since ON/OFF sWitching of the generation pro
cessing operation at the means for generating Wide dynamic
an object involving a motion oWing to the fact that the
motion in the object can in effect be reduced by using a
range, synthesiZed image is thus controlled based on infor
higher shutter speed.
mation set in the image pickup apparatus before the taking
[0025] In a sixth aspect of the invention, an image pickup
apparatus according to the fourth aspect includes a shutter
speed/ diaphragm control means. In addition, the determina
tion means thereof includes means for detecting motion in
the object image so that, When a motion in the object image
has been detected by the motion detection means, images are
taken by using a higher shutter speed to perform the gen
eration processing of synthesiZed image.
of image, an image pickup apparatus can be achieved as
capable of readily taking images in a manner suitable for the
generation of Wide dynamic range, synthesiZed image With
out requiring detection of the conditions for the generation
of Wide dynamic range, synthesiZed image from taken
images. The above object is thereby accomplished.
[0033]
It is a further object of the present invention to
provide an image pickup apparatus in Which it is possible to
make a determination based on the shutter speed and lens
[0026] Since the generation processing of synthesiZed
Zoom information as to Whether or not a suitable synthesiZed
image is thus performed at a higher shutter speed When a
motion in the object image has been detected, the motion of
image can be generated.
the object in image signals corresponding to a plurality of
frames can be reduced. Since the motion of the object can in
effect be reduced even of an object involving a motion by
[0034]
In a ninth aspect of the invention, information set
in the image pickup apparatus according to the eighth aspect
is shutter speed information and/or lens Zoom information
using the higher shutter speed, it becomes possible to readily
generate a Wide dynamic range, synthesiZed image. The
above object is thereby accomplished.
and the determination means makes a determination based
on the shutter speed and/or lens Zoom information.
[0027]
can be performed in linkage With the shutter speed and lens
It is a further object of the invention to provide an
image pickup apparatus capable of generating a suitable
Wide dynamic range, synthesiZed image by detecting a
backlighted condition.
[0028]
In a seventh aspect of the invention, the determi
[0035] By constructing in this manner, ON/OFF sWitching
of the generation processing operation of synthesiZed image
Zoom information. When the shutter speed is sloW or When
the Zoom ratio is high, the generation processing operation
of synthesiZed image can be turned OFF to avoid a failure
in the synthesiZed image Which occurs due to a movement
nation means of the image pickup apparatus according to the
of the object. The above object is thereby accomplished.
fourth aspect includes an AB detection circuit and the
[0036]
generation processing of synthesiZed image is performed
When a backlighted condition has been detected by the AE
detection circuit.
[0029] Upon the detection of a backlighted condition at
the AE detection circuit, since the condition for Wide
dynamic range prevails at the time of the backlighted
condition, a suitable synthesiZed image can be obtained by
performing the generation processing of Wide dynamic
range, synthesiZed image. The above object is thereby
accomplished.
[0030] It is a further object of the present invention to
provide an image pickup apparatus in Which images can be
readily taken in a manner suitable for the generation of Wide
dynamic range, synthesiZed image Without requiring detec
It is a further object of the present invention to
provide an image pickup apparatus having a function for
generating Wide dynamic range, synthesiZed image by syn
thesiZing a plurality of image signals of different exposure
amounts, in Which a suitable synthesiZed image is alWays
obtained even When a shift from set exposure amounts has
occurred.
[0037]
In a tenth aspect of the invention, there is provided
an image pickup apparatus including: image pickup means
capable of taking images of the same object at a plurality of
different exposure amounts to generate image signals cor
responding to a plurality of frames at different exposure
amounts; and means for generating Wide dynamic range,
synthesiZed image by synthesiZing image signals corre
sponding to a plurality of frames at different exposure
amounts obtained by the image pickup means. It further
tion of the conditions for the generation of Wide dynamic
includes: means for correcting exposure amounts of the
range, synthesiZed image from taken images.
image signals corresponding to a plurality of frames of
[0031]
performing ON/OFF sWitching of the correction operation at
In an eighth aspect of the invention, there is pro
different exposure amounts; and sWitching control means for
vided an image pickup apparatus including: image pickup
the exposure amount correction means based on an image
means capable of taking images of the same object at a
plurality of different exposure amounts to generate image
signals corresponding to a plurality of frames at different
exposure amounts; and means for generating Wide dynamic
taking mode set at the image pickup apparatus.
[0038] By thus providing the means for correcting expo
sure amounts of image signals and ON/OFF sWitching
range, synthesiZed image by synthesiZing image signals
corresponding to a plurality of frames at different exposure
amounts obtained by the image pickup means. It further
includes means for determining Whether or not a suitable
synthesiZed image can be obtained on the basis of informa
control means based on the image taking mode at the
exposure amount correction means, the exposure amounts
can be corrected to obtain a suitable synthesiZed image only
When there is a possibility of shift from the set exposure
amounts of the image signals corresponding to a plurality of
frames of different exposure amounts. The above object is
tion set in the image pickup apparatus before the taking of
image; and means for controlling ON/OFF sWitching of the
generation processing operation at the means for generating
thereby accomplished.
synthesiZed image based on an output of the determination
image pickup apparatus in Which a suitable synthesiZed
means.
image can be obtained even When using a strobe.
[0039]
It is a further object of the invention to provide an
US 2007/0146538 A1
[0040] In an eleventh aspect of the invention, the image
pickup apparatus according to the tenth aspect further
includes strobe emission means. The switching control
means controls sWitching so that the exposure amount
Jun. 28, 2007
remaining capacity of the recording medium is not enough
for recording image signals corresponding to a plurality of
frames. The above object is thereby accomplished.
[0048]
It is a further object of the present invention to
correction means is caused to operate When the apparatus
has been set to a strobe image taking mode.
provide an image pickup apparatus in Which, When image
[0041]
a Wide dynamic range, synthesized image are to be recorded
in a recording medium, synthesizing process thereof can be
performed by treating them in a similar manner as image
signals corresponding to an ordinary one frame.
While a variance tends to occur in the emission
amount of a strobe, a suitable synthesized image for example
Without a discontinuity in gradation can be generated even
When images are taken by using strobe, since the exposure
amount correction means is forced to operate at the time of
the strobe image taking mode. The above object is thereby
signals corresponding to a plurality of frames for generating
[0049]
In a fourteenth aspect of the invention, there is
accomplished.
provided an image pickup apparatus Which includes: image
pickup means capable of taking images of the same object
[0042]
at a plurality of different exposure amounts to generate
It is a further object of the invention to provide an
image synthesizing system capable of generating Wide
dynamic range, synthesized image in Which image taking
interval can be reduced Without requiring a complicated
processing at the image pickup apparatus thereof.
[0043]
In a tWelfth aspect of the invention, there is pro
vided an image synthesizing system, including: an image
pickup apparatus having image pickup means capable of
taking images of the same object at a plurality of different
image signals corresponding to a plurality of frames at
different expo sure amounts; and recording means for record
ing as a single ?le to a recording medium the image signals
corresponding to a plurality of frames obtained by the image
pickup means and synthesizing information to be used in
generating Wide dynamic range, synthesized image by syn
thesizing the image signals corresponding to the plurality of
frames.
exposure amounts to generate image signals corresponding
[0050] Since image signals corresponding to a plurality of
to a plurality of frames at different exposure amounts and
recording means for recording as a single ?le to a recording
frames and the synthesizing information thereof are
recorded as a single ?le to the recording medium, it is
medium the image signals corresponding to a plurality of
frames obtained by the image pickup means and synthesiz
ing information to be used in generating a Wide dynamic
possible to perform synthesizing process by treating them
range, synthesized image by synthesizing the image signals
plished.
corresponding to the plurality of frames; and an external
synthesizing apparatus having synthesizing means for syn
thesizing the image signals corresponding to a plurality of
frames Within a ?le recorded in the recording medium based
on the synthesizing information in the ?le.
[0044] By thus performing synthesizing process at an
external synthesizing apparatus externally connected to the
image pickup apparatus, it becomes unnecessary to perform
a complicated synthesizing process Within the image pickup
apparatus. Time for processing of image taking is shortened
and it becomes possible to reduce the image taking interval.
The above object is thereby accomplished.
[0045]
It is a further object of the present invention to
provide an image synthesizing system in Which image
signals corresponding to one frame can be recorded When
similarly as in the case of image signals corresponding to an
ordinary single frame. The above object is thereby accom
[0051]
It is a further object of the present invention to
provide an external synthesizing apparatus in Which, When
a Wide dynamic range, synthesized image is generated by
synthesizing image signals corresponding to a plurality of
frames, the synthesized image can be generated similarly as
an ordinary single image signal Without a special operation.
[0052] In a ?fteenth aspect of the invention, there is
provided an external synthesizing apparatus including syn
thesizing means for reading a ?le out from a recording
medium, said ?le being recorded in said recording medium
as a single ?le recording image signals corresponding to a
plurality of frames of different exposure amounts obtained
by taking images of the same object at a plurality of different
exposure amounts and synthesizing information to be used
in generating Wide dynamic range, synthesized image by
synthesizing the image signals corresponding to the plurality
the remaining capacity of the recording medium is less than
that for recording image signals corresponding to a plurality
of frames, said synthesizing means generating Wide dynamic
of frames.
range, synthesized image by synthesizing the image signals
[0046] In a thirteenth aspect of the invention, the image
synthesizing system according to the tWelfth aspect further
includes detection means for detecting the remaining capac
based on the synthesizing information contained in the ?le.
ity of the recording medium. The remaining capacity of the
recording medium is detected by the detection means and, if
not enough remaining capacity left in the recording medium
for recording image signals corresponding to a plurality of
frames of different exposure amounts, image signals corre
sponding to one frame is obtained by taking image of the
object at a predetermined exposure amount by the image
pickup means and such image signals corresponding to one
frame is recorded to the recording medium by the recording
corresponding to the plurality of frames contained in the ?le
[0053] Since image signals corresponding to a plurality of
frames and synthesizing information thereof are thus
recorded as a single ?le in a recording medium from Which
said ?le is read out to effect synthesizing process, a Wide
dynamic range synthesized image can be outputted similarly
as in the case of an ordinary taken image Without a special
operation. The above object is thereby accomplished.
[0054] It is a further object of the invention to provide a
recording medium recorded in Which includes a program
capable of readily performing synthesis of image signals
means.
corresponding to a plurality of frames obtained by taking
[0047] By such construction, an ordinary image taking
images of the same object at a plurality of different exposure
corresponding to one frame can be performed even When the
amounts.
US 2007/0146538 A1
[0055] A recording medium recording a program for per
forming synthesis of image signals corresponding to a
plurality of frames obtained by taking images of the same
object at a plurality of different exposure amounts is pro
vided in accordance With a sixteenth aspect of the invention,
the recording medium recording, as a single ?le, the image
signals corresponding to the plurality of frames of different
exposure amounts and synthesiZing information to be used
in generating a Wide dynamic range, synthesiZed image by
synthesiZing the image signals corresponding to the plurality
of frames and at the same time recording a program for
achieving a function for reading out the ?le to synthesiZe the
image signals corresponding to the plurality of frames
contained in the ?le based on the synthesiZing information
contained in the ?le.
[0056] By constructing in this manner, a recording
medium can be provided as that recording a program by
Which synthesis can be readily performed of image signals
corresponding to a plurality of frames obtained by taking
images of the same object at a plurality of different exposure
amounts. The above object is thereby accomplished.
BRIEF DESCRIPTION OF THE DRAWING
[0057] FIG. 1 is a block diagram shoWing an overall
construction of a ?rst embodiment of the image pickup
apparatus according to the present invention.
[0058] FIG. 2 is a block diagram shoWing an example of
construction of the camera signal processing circuit in the
embodiment shoWn in FIG. 1.
[0059] FIGS. 3A to 3D illustrate the manner of detecting
motion at the motion detecting section in the ?rst embodi
ment.
[0060]
FIG. 4 explains the manner of correcting a motion
by the motion correcting circuit.
[0061] FIGS. 5A, 5B, 5C are ?oWcharts explaining cam
era control operation for performing ON/OFF sWitching of
the generation processing of Wide dynamic range, synthe
siZed image based on previously taken image data.
[0062]
FIG. 6 is a block diagram shoWing main portions of
Jun. 28, 2007
[0071] FIG. 15 shoWs the ?le structuring of image data
and synthesiZing information to be used in their synthesiZing
process.
[0072] FIG. 16 is a ?owchart explaining the processing
operation in causing an externally connected personal com
puter to perform synthesiZing process.
DESCRIPTION OF THE PREFERRED
EMBODIMENTS
[0073] Some embodiments of the present invention Will
noW be described. First, a description Will be given beloW by
Way of a block diagram shoWn in FIG. 1 With respect to an
overall construction of a ?rst embodiment of the electronic
camera to Which the present invention is applied. Referring
to FIG. 1, denoted by numeral 1 is a single-plate (sensor)
color CCD image pickup device for photoelectrically con
verting an optical signal into an electrical signal. It includes
an electronic shutter function. An object light is inputted to
CCD image pickup device 1 through a lens 2 and a stop/
shutter mechanism 3. An output of CCD image pickup
device 1 is ampli?ed at an ampli?er 4 after removed of noise
for example at a correlation double sampling circuit. An
analog-to-digital converter 5 converts the output of the
ampli?er 4 outputted as analog data into digital data. A
camera signal processing circuit 6 processes signals from
CCD image pickup device 1 as a video data. Those denoted
by numeral 7 include AF detection circuit for extracting AF
(autofocus) information to control focus, an AE detection
circuit for extracting AE (auto exposure) information to
control exposure, and AWB detection circuit for extracting
AWB (auto White balance) information to set White balance,
by using image pickup signals etc., from the CCD image
pickup device 1 prior to the main taking of image. Output
signals from the AF, AE, AWB detection circuit 7 supplies
through CPU 8 AF information to lens 2, AE information to
stop/shutter mechanism 3, and AWB information to the
camera signal processing circuit 6.
[0074] Numeral 9 denotes a compression circuit (JPEG)
for compressing the amount of data. The image data com
pressed at the compression circuit 9 is recorded at memory
card 15 through memory card I/F 14. Memory controller 10
and DRAM 11 are used as Working memory When perform
a second embodiment of the present invention.
ing color processing, etc., of video data. A display circuit 12
[0063] FIG. 7 is a block diagram shoWing an example of
construction of camera signal processing circuit of the
image taking conditions by reading out and displaying data
second embodiment shoWn in FIG. 6.
[0064] FIG. 8 shoWs the manner of AF processing at an
external AF circuit.
[0065]
FIG. 9 is a block diagram shoWing main portions of
and LCD display unit 13 are used for example to con?rm the
recorded at the memory card 15. Denoted by numeral 16 is
a personal computer I/ F for transferring data recorded on the
memory card 15 to a personal computer 17. Further, denoted
by numeral 19 is a printer for providing a print output of
image pickup data recorded on the memory card 15 and 18
is a printer I/F to be used in transferring data to the printer
a modi?cation of the second embodiment shoWn in FIG. 6.
19. It should be noted that, in FIG. 1, numeral 20 denotes a
[0066] FIG. 10 shoWs the manner ofAE processing at an
external AE circuit.
timing generator Which generates timing pulse for driving
[0067] FIG. 11 is a block diagram shoWing main portions
of another modi?cation of the second embodiment.
[0068] FIG. 12 is a block diagram shoWing main portions
of a third embodiment of the present invention.
[0069]
FIG. 13 shoWs a program diagram of shutter speed/
the CCD image pickup device 1. It drives the CCD image
pickup device 1 according to control of CPU 8. Numeral 21
denotes a strobe mechanism Which is controlled through
CPU 8 by AE information to be obtained before the main
taking of image and effects control as to Whether strobe is to
be emitted or not and control of light amount of the strobe
emission. Numeral 22 denotes input keys of CPU by Which
setting of various types of image taking mode, driving of
aperture stop.
trigger sWitch, etc., can be performed.
[0070]
[0075] Operation of an electronic camera having the above
construction is summariZed as folloWs. Image pickup signals
amount.
FIG. 14 explains the manner of correcting exposure
US 2007/0146538 A1
generated by means of tWo times of image taking under
different exposure conditions by using the CCD image
pickup device 1 are respectively converted into digital
signals at the analog-to-digital converter 5 and are subjected
to a predetermined synthesizing operation processing at the
camera signal processing circuit 6 to obtain a single syn
thesiZed image data Which has a Wide dynamic range as a
Whole. The synthesiZed image data is compressed at the
compression circuit 9 and is recorded to the memory card
15.
Jun. 28, 2007
[0078] Actual motion detecting techniques to be per
formed at the motion detecting section 8A provided Within
CPU 8 Will noW be described by Way of FIGS. 3A to 3D.
There are tWo methods for detecting motion from previously
taken tWo frames of image data. In a ?rst method, a
difference is obtained of a short-time exposure image data as
shoWn in FIG. 3A (object 3111 having a lack of detail at loW
level portion and background 31b taken at a suitable expo
sure in this short-time exposure image data) from a long
time exposure image data as shoWn in FIG. 3B (object 3211
taken at a suitable exposure but background 32b lacking
[0076] A construction of the camera signal processing
detail at its high level portion in this long-time exposure
circuit 6 in an embodiment shoWn in FIG. 1 Will noW be
image data). If no motion occurs in the object, such differ
described by Way of the block diagram of FIG. 2. FIG. 2
includes: 6-1, SE memory for storing short-time exposure
the other hand, if there is a motion, difference at the portion
image data; 6-2, LE memory for storing long-time exposure
image data; 6-3, a multiplier for multiplying a short-time
exposure image data read out from the SE memory 6-1 by
an exposure amount ratio A (=LE/SE) of the short-time
exposure image and long-time exposure image; 6-4, a syn
thesiZing circuit for forming by means of synthesis a Wide
dynamic range, synthesiZed image from the short-time expo
sure image data after the multiplication and the long-time
exposure image data read out from the LE memory 6-2; 6-5,
a compression circuit for compressing the synthesiZed image
obtained at the synthesiZing circuit 6-4; 6-6, a signal pro
ence becomes Zero and an absence of motion is detected. On
of occurrence of motion does not become Zero as shoWn in
FIG. 3C and a motion is thereby detected. It should be noted
that, When obtaining difference betWeen the tWo image data
of different exposure amounts, the short-time exposure
image data SE is multiplied by the exposure amount ratio A
and its difference (LE—SE><A) from the long-time exposure
image data LE is obtained. Referring to FIG. 3C, the slants
represent the portion Where the difference is “+” While the
dotted region represents the portion Where the difference is
[0079] In another method for detecting motion, it is
cessing circuit for subjecting the long-time exposure image
detected from ratio of the tWo image data of different
data read out from the LE memory 6-2 to such processing as
exposure amounts. The exposure amount ratio of the tWo
y-correction and edge enhancement; and 6-7, a selector for
providing an output by sWitching based on control signal
from CPU 8 betWeen the synthesiZed image from the
synthesiZing circuit 6-4 and the long-time exposure image
data having been subjected to signal processing at the signal
processing circuit 6-6. Further, denoted by numeral 8A is a
motion detecting section provided internally of CPU 8, for
detecting motion in the object based on the short-time
exposure image data and long-time exposure image data
read out from the SE memory 6-1 and LE memory 6-2.
SWitching of the selector 6-7 is controlled on the basis of the
output of the motion detecting section 8A.
[0077]
A description Will noW be given With respect to an
operation of thus constructed camera signal processing
circuit 6. First, short-time exposure image data and a long
time exposure image data of the same object taken at the
CCD image pickup device 1 are temporarily stored to SE
memory 6-1 and LE memory 6-2. The image data are then
transmitted into CPU 8 from the tWo memories 6-1, 6-2. A
motion of the object is detected at the motion detecting
section 8Abased on the tWo image data of different exposure
amounts. If no motion is detected at the motion detecting
section 8A, a Wide dynamic range, synthesiZed image syn
thesiZed at the synthesiZing circuit 6-4 based on the-image
data read out from the tWo memories 6-1, 6-2 and com
pressed at the compression circuit 6-5 is outputted through
the selector 6-7 Which is controlled by CPU 8. On the other
image data, A=LE/SE, is represented by 0t~A. When a
motion as shoWn in FIG. 3D occurs, the slant portion
represents a region Where 0t>l and the dotted portion
represents a region Where 0t<l, both indicating the moving
portion. It is determined as an occurrence of motion of the
object When these regions are detected.
[0080] Amodi?cation of the ?rst embodiment Will noW be
described. In this modi?cation, When the amount of motion
detected at the motion detecting section 8A is relatively
small and is determined as Within a range of alloWable
amount for correction by CPU 8, the motion is corrected and
synthesiZing process for Wide dynamic range, synthesiZed
image is performed. The motion correction circuit in this
case is constructed as included in the synthesiZing circuit
6-4.
[0081] In the method of correction by the motion correc
tion circuit, if a region indicated by X is the region Where
motion is to be corrected When regions along a horizontal
direction are taken into consideration as shoWn in FIG. 4,
values are used of pixels A, B in regions Without an
occurrence of motion, at a suitable distance d for performing
correction from the tWo ends of the region X to be corrected
of motion. The A, B pixels are determined as at tWo ends and
the motion is corrected by interpolating the motion correc
tion region X so as to uniformly change pixel values
betWeen the tWo.
data read out from the LE memory 6-2 and subjected to
[0082] It should be noted that, since the region (object
portion) Without lacking detail in its highlighted portion in
signal processing at the signal processing circuit 6-6 is
outputted through the selector 6-7. Amotion is thus detected
the long-time exposure image is used as it is for synthesis in
generating a synthesiZed image, the “—” region Where
on the basis of previously taken image data so that it is
motion occurs as shoWn in FIG. 3C is not corrected and only
hand, if a motion is detected, the long-time exposure image
possible to prevent a failed, synthesiZed image from being
the “+” region [i.e., region of LE—(SE><A)>0] is subjected to
outputted When the object is moved at the time of taking tWo
the motion correction. Further, as shoWn in FIG. 3D, since
the region of 0t<l is used as it is in the synthesiZing process,
images at different exposure amounts.
US 2007/0146538 A1
a motion correction thereof is not performed and only the
region of (X>l is corrected of motion.
[0083]
Shown in FIGS. 5A to 5C are ?owcharts collec
tively representing a camera control operation in the above
embodiment for effecting ON/OFF switching of the output
of wide dynamic range, synthesized image on the basis of
motion in previously taken image data, etc. FIG. 5A is the
?owchart of a broader term of this embodiment showing
camera control operation where ON/OFF switching of the
synthesiZed image output is performed by determining based
on previously taken image data whether or not the wide
dynamic range, synthesiZed image is suitable. At ?rst, in this
case, AE, AP control is performed based on AE, AP infor
mation obtained before the main taking of image (step S1).
Jun. 28, 2007
object on the basis of AF signals from the external AF circuit
23. Further, as shown in FIG. 7, the only frame memory
included in the camera signal processing circuit 6' is SE
memory 6-1 for storing short-time exposure image data. The
short-time exposure image data is taken into SE memory
6-1. The short-time exposure image data is read out from SE
memory 6-1 in accordance with timing of the long-time
exposure image data and, after multiplication processing
performed based on the exposure amount ratio, synthesiZing
process is performed at the synthesiZing circuit 6-4 and
compression processing is performed at the compression
circuit 6-5. Furthermore, the AE, AP, AWB detection circuit
7 in the ?rst embodiment is replaced by an AE, AWB
detection circuit 7'.
An image is then taken twice under different exposure
conditions (noted as SL image taking) to generate two image
[0087] Those which may be used as the external AF circuit
23 for example includes an AP sensor capable of ?nding
data (step S2). It is then determined whether wide dynamic
range, synthesiZed image (noted as SL synthesiZed image)
range of a plurality of points (?ve points in the illustrated
will be suitable or not on the basis of the two image data
performed twice and range-?nding information is inputted to
the motion detecting section 8B which is provided internally
(step S3). If the wide dynamic range, synthesiZed image is
suitable, the wide dynamic range, synthesiZed image is
outputted (step S4). If the synthesiZed image is unsuitable,
example) on the object as shown in FIG. 8. AF operation is
the image data of long-time exposure (noted as LE image)
of CPU 8. It is determined at the motion detection circuit 8B
as that the object is moving when a change occurs in
distance information obtained by the two times of AF
is outputted (step S5).
operation for any one of the plurality of range-?nding points.
[0084]
FIG. 5B is the ?owchart showing a camera control
operation in the case of detecting motion of the object from
two image data under different exposure conditions to
determine whether wide dynamic range, synthesized image
The image is taken only at a long-time exposure based on the
second distance information and, without performing the
generation processing of synthesiZed image, only the image
data of the long-time exposure is outputted.
will be suitable or not. In particular, after generating two
image data of different exposure amounts at the same step,
motion of the object is detected on the basis of the two image
[0088] In other words, in the second embodiment, motion
of the object is detected at the motion detecting section 8B
internally provided of CPU, based on AF signals from the
data (step S11). If motion of the object is not detected, a wide
external AF circuit. If no motion is detected, the selector 6-7
dynamic range, synthesiZed image is outputted (step S12). If
is previously switched at the camera signal processing
motion of the object is detected, the wide dynamic range,
synthesiZed image is determined as unsuitable and the
circuit 6' so as to output a synthesiZed and compressed wide
dynamic range image. If motion is detected, on the other
long-time exposure image data is outputted (step S13).
hand, the selector 6-7 is previously switched so that a
[0085]
long-time exposure image data taken by the second AF
information is outputted to prevent a synthesiZed image
failed due to motion from being outputted.
FIG. 5C is a ?owchart showing a camera control
operation where, even when motion has been detected by
detecting motion of the object from two image data under
different exposure conditions, motion is corrected if the
amounts and, if motion is detected, it is determined whether
[0089] Since ON/OFF switching of the outputting of syn
thesiZed image is thus previously performed based on AF
information obtained before the taking of image from an
external AF circuit, it is not necessary to drive CCD image
pickup device in order to perform ON/OFF switching con
the amount of the motion is within an allowable range for
trol of the synthesiZed image output, thereby reducing
amount of motion is within an allowable range for correc
tion. In particular, motion of the object is similarly detected
on the basis of the two image data of different exposure
correction or not (step S14). If, then, the amount of motion
dissipation power. Further, it is not necessary to detect a
is smaller than an allowable amount for correction, a wide
motion on the basis of image data after the taking of image
dynamic range, synthesiZed image after corrected of motion
is outputted (step S15). If the amount of motion is greater
than the allowable amount for correction, the image data of
the long-time exposure is outputted (step S16).
[0086] A second embodiment will now be described. In
this embodiment, camera control including ON/OFF switch
so that images can be taken in a manner suitable for the
generation of wide dynamic range, synthesiZed image from
the beginning.
[0090] A modi?cation of the present embodiment will
now be described. In the above second embodiment, the
obtained before the taking of image. FIG. 6 is a block
external AF circuit is provided and motion of the object is
detected on the basis of AF signals. In this modi?cation,
however, an external AE circuit 24 is provided as shown in
FIG. 9 instead of the external AF circuit. A motion of the
diagram showing main portions of the second embodiment,
object is detected by a motion detecting section 8C provided
where like components as in the ?rst embodiment shown in
FIG. 1 are denoted by like reference numerals. This is
different from the ?rst embodiment shown in FIG. 1 in that
an external AF circuit 23 is provided and CPU 8 includes a
in CPU 8 on the basis of two times of AE signals from the
ing of the generation and outputting of wide dynamic range,
synthesiZed image is performed on the basis of information
motion detecting section 8B for detecting motion of the
external AE circuit 24. In a similar manner as in the second
embodiment based on such motion detection signals, if no
motion is detected, the selector 6-7 of the camera signal
processing circuit 6' shown in FIG. 7 is previously switched
US 2007/0146538 A1
so as to output a Wide dynamic range, synthesized image
Which has been synthesized and compressed. If a motion is
detected, the selector 6-7 at the camera signal processing
circuit 6' is also previously sWitched so that image data of
long-time exposure based on the second AE information is
outputted. It should be noted that denoted by 7" in FIG. 9 is
Jun. 28, 2007
memory to detect motion of the object by using such knoWn
techniques as frame correlation. If a motion has been
detected at the motion detection circuit 26, image is taken
tWice in a short time period With causing the strobe 21
through CPU 8 to emit tWice by changing emission amount
in such a manner as to correspond to an exposure amount
an AP, AWB detection circuit.
ratio for obtaining a Wide dynamic range image. A Wide
[0091] Those Which may be used as the external AE circuit
24 for example includes an AB sensor capable of photometry
performing synthesiZing process. Further, if motion is not
by dividing the object into a plurality of portions as shoWn
in FIG. 10. AE operation is performed tWice and exposure
information is inputted to the motion detecting section 8C
internally located at CPU 8. It is determined at the motion
detecting section 8C as that the object is moving and image
is taken of long-time exposure by the second exposure
information When a change in exposure information occurs
in any one of the plurality of divisional photometric regions
betWeen the tWo times of AE operation.
[0092] As the above, in this modi?cation, too, motion of
the object is detected at the motion detecting section 8C
internally provided of CPU on the basis of AE information
from the external AE circuit. When no motion is detected,
the selector 6-7 of the camera signal processing circuit 6' is
previously sWitched so that a Wide dynamic range, synthe
siZed image after synthesis and compression is outputted. If
a motion is detected, on the other hand, the selector 6-7 is
dynamic range, synthesiZed image is then generated by
detected, image is taken tWice at different exposure amounts
Without using a forced emission of the strobe 21 to perform
the generation processing of Wide dynamic range, synthe
siZed image. Accordingly, in this embodiment, the camera
signal processing circuit does not require a selector as that
shoWn in FIG. 2.
[0096] The generation processing of Wide dynamic range,
synthesiZed images is performed by thus using a forced
strobe mode correspondingly to the motion of the object.
Since, thereby, images can be taken With stopping the
motion of the object for the tWo times of strobe image taking
in a short time period, it is possible to prevent a failed
synthesiZed image from being outputted.
[0097]
This embodiment has been shoWn as that in Which
motion of the object is detected by using image pickup data
from the CCD image pickup device. As shoWn in FIG. 6 or
9, hoWever, it is also possible to detect motion at a motion
previously sWitched so that image data of long-time expo
sure taken by the second AE information is outputted to
prevent a synthesiZed image failed due to motion from being
detecting section provided internally of CPU from AP, AE
information obtained before the taking of image by using an
outputted. Accordingly, since ON/OFF sWitching of the
synthesiZed image output can be performed Without driving
the image pickup device, it is possible to reduce dissipation
the generation processing of synthesiZed image to be per
poWer and it is not necessary to detect motion on the basis
of image data after the taking of image so that images can
external AF circuit or external AE circuit, thereby causing
formed in a forced strobe mode.
[0098] A modi?cation of the third embodiment Will noW
be described. In this modi?cation, When a motion of the
this modi?cation, as shoWn in FIG. 11, a camera shake
object has been detected at the motion detecting circuit 26,
a higher shutter speed is used instead of using a strobe
emission. Exposure time is thereby reduced for both the
short-time exposure image taking and the long-time expo
sure image taking so that the generation processing of Wide
detecting circuit 25 for example using an acceleration sensor
is provided so that, When camera shake is detected, an image
tWo images at different exposure amounts in a short time
taking for the generation of synthesiZed image is not per
period.
be taken in a manner suitable for the generation processing
of synthesiZed image from the beginning.
[0093]
A further modi?cation Will noW be described. In
formed and image is taken aneW. In other Words, an occur
rence of camera shake causes a difference betWeen the tWo
image data of different exposure amounts; if these are
synthesiZed, an unsuitable synthesiZed image results.
Accordingly, image is taken aneW in case of camera shake
so that it is possible to prevent a failed synthesiZed image
from being outputted.
[0094]
A third embodiment Will noW be described. FIG. 12
is a block diagram shoWing main portions of the third
embodiment. In this embodiment, a motion detection circuit
26 is provided so that motion of the object is previously
detected on the basis of image pickup data Which is obtained
from CCD image pickup device 1 before the main taking of
image. When a motion has been detected, a Wide dynamic
range, synthesiZed image is generated by taking image tWice
at different exposure amounts (emission amount) by using a
forced strobe emission.
[0095] The motion detection circuit 26 includes a memory
for image data corresponding to tWo frames so that tWo
image data of the same object previously obtained from
CCD image pickup device 1 are altemately stored to the
dynamic range, synthesiZed image is performed by taking
[0099] In the case Where images are taken by using a
higher shutter speed as the above, a program shift for
Widening the aperture stop corresponding to the shutter
speed is performed in order to achieve the same exposure
amount even With the reduced exposure time due to the
higher shutter speed. Speci?cally, in the program diagram of
shutter speed/aperture stop as shoWn in FIG. 13, if a motion
has been detected at point “A” Where shutter speed is 1/30 and
aperture value is F8, a program shift is made for example by
shifting to point “B” Where shutter speed is 1/250 and aperture
is Widened to P28. In this manner, the generation processing
of Wide dynamic range, synthesiZed image is performed by
taking image tWice at different exposure amounts by using
a higher shutter speed. It is thereby possible to make smaller
the motion of the object so that a synthesiZed image With
lessened failure can be outputted.
[0100] A fourth embodiment Will noW be described. In this
embodiment, When the object has been detected as in a
backlighted condition by means of AE information previ
ously obtained on the basis of image data from CCD image
pickup device before the taking of image or AE information
US 2007/0146538 A1
Jun. 28, 2007
obtained from an external AE circuit, the generation pro
in setting exposure, it is most likely the case Where an
cessing of Wide dynamic range, synthesized image is per
formed by taking image tWice at different exposure amounts,
tometry because of a backlighted condition. Therefore, if a
since image data in a backlighted condition tends to have a
Wider dynamic range. In this case, too, since ON/OFF
generation of a Wide dynamic range, synthesiZed image.
sWitching of the generation processing of Wide dynamic
range, synthesiZed image is performed on the basis of AE
information previously obtained before the taking of image,
the images can be taken in a manner suitable for the
generation of Wide dynamic range, synthesiZed image from
the beginning.
[0101]
A ?fth embodiment Will noW be described. In this
embodiment, camera control including ON/OFF sWitching
of the generation processing of Wide dynamic range, syn
thesiZed image is performed on the basis of such camera
setting information as the image taking conditions set to the
image pickup apparatus (camera) before the taking of image.
Although the fundamental construction of this embodiment
is similar to that of the ?rst embodiment shoWn in FIG. 1, its
camera signal processing circuit is of similar construction as
that used in the second embodiment shoWn in FIG. 7. Here,
camera control including ON/OFF sWitching of the genera
tion processing of Wide dynamic range, synthesiZed image is
performed on the basis of various camera setting informa
tion to be set from the input key 22 of CPU 8. By thus
causing camera control including the ON/OFF sWitching of
Wide dynamic range, synthesiZed image to be performed on
the basis of information set to the camera before the taking
of image, it becomes unnecessary to detect motion, etc.,
based on image pickup data after the taking of image and
images can be taken in a manner suitable for the generation
of Wide dynamic range, synthesiZed image from the begin
ning, since ON/OFF sWitching of the generation processing
of synthesiZed image is previously performed.
[0102]
In a ?rst manner of this embodiment, images are
taken for the generation of Wide dynamic range, synthesiZed
image and the generation processing of synthesiZed image is
performed When taking image in a strobe mode Where strobe
emission is used in combination, thereby linking ON/OFF
sWitching of the synthesizing process to the setting of a
attempt is made to take image With performing spot pho
spot photometry mode is set, images are taken for the
[0105] Further, When an exposure correction is effected
manually in a camera having a manual exposure correcting
function, since a manual exposure correction is most likely
attempted in the case Where the scene to be photographed is
in a backlighted condition or the dynamic range of the object
is large, images are taken for the generation of Wide dynamic
range, synthesiZed image at the time of manually corrected
exposure.
[0106]
In a fourth manner of this embodiment, ON/OFF
sWitching of the generation processing of Wide dynamic
range, synthesiZed image is linked to the image taking mode.
With a camera having a consecutive photographing function,
for example, since the taking of image for the generation of
Wide dynamic range, synthesiZed image cannot be effected
in a consecutive taking mode, the generation processing of
synthesiZed image is sWitched to OFF When the consecutive
taking mode has been set. Further, since too much motion is
not likely to be involved in a scenery (distant vieW) photo
graphing mode, the generation processing of synthesiZed
image is turned ON When such mode has been set. Further
more, since a large motion of the object is likely in a sports
photographing mode, the generation processing of synthe
siZed image is turned OFF When such mode has been set.
Moreover, When a macro strobe photographing mode has
been set, the generation processing of synthesiZed image is
turned ON similarly to the strobe mode.
[0107] Since, in this manner, the generation processing of
synthesiZed image is thus performed only When an image
taking mode suitable for the generation of Wide dynamic
range, synthesiZed image has been set, it is possible to obtain
a Wide dynamic range, synthesiZed image Without failure.
[0108]
A sixth embodiment Will noW be described. In
general, image is taken tWice by previously setting exposure
amount ratio thereof in a camera having a function for
strobe mode. In particular, even in the case of an object With
generating Wide dynamic range, synthesiZed image. The
motion or the like, it is possible When taking image in a
strobe mode that the motion of the object be stopped and a
Wide dynamic range, synthesiZed image Without a failure be
obtained by causing strobe emission so as to take image
exposure amount ratio of the tWo images actually taken,
tWice at different exposure amounts in a short time period.
[0103]
In a second manner of this embodiment, ON/OFF
sWitching of the generation processing of Wide dynamic
range, synthesiZed image is linked to the shutter speed to be
set. In particular, since a sloWer shutter speed results in a
greater motion of the object, the generation processing of
Wide dynamic range, synthesiZed image is turned OFF. The
hoWever, is not necessarily the same as the set value. If they
are synthesiZed in such case, there is a disadvantage that a
discontinuity in gradation tends to occur at the seam
betWeen the tWo images. This embodiment intends to miti
gate this problem.
[0109] Speci?cally, in this embodiment, a camera having
a function for generating Wide dynamic range, synthesiZed
image includes exposure amount correction means and
ON/OFF sWitching of the exposure amount correction cor
responding to shift in the exposure amount ratio is controlled
shutter speed serving as a basis for ON/OFF sWitching of
synthesiZing process someWhat varies in a camera having
Zoom depending on Zoom ratio. In particular, since a higher
Zoom ratio magni?es the motion, the set shutter speed for
tion, since exposure amount correction is performed only
When an image taking mode requiring exposure amount
turning OFF the synthesiZed image generation processing is
correction has been set, it is not necessary to alWays detect
shifted toWard a higher shutter speed.
exposure amount to determine Whether an exposure amount
[0104]
In a third manner of this embodiment, ON/OFF
sWitching of the generation processing of Wide dynamic
range, synthesiZed image is linked to the setting of an
exposure. If, for example, the user effects spot photometry
on the basis of setting of an image taking mode Which may
cause a shift in the exposure amount ratio. By this construc
correction is required or not.
[0110] In particular, an exposure amount correction is
performed When a strobe image taking mode has been set. In
other Words, the exposure amount in taking image by using
US 2007/0146538 A1
Jun. 28, 2007
strobe depends not on the shutter speed but on the emission
amount of the strobe. Since the strobe emission amount
cannot so accurately be controlled, a considerable variance
may occur in the exposure amount. Accordingly, correction
of exposure amount is performed When a strobe emission
mode has been set.
to generate image data corresponding to tWo frames. Such
[0111] The correcting operation of the exposure amount is
header 41, short-time exposure image data 42 and long-time
performed as folloWs. Image data With a smaller exposure
amount and image data With a greater exposure amount
obtained by tWo times of image taking using strobe emission
are stored to the tWo memories, i.e., SE memory and LE
memory of the camera signal processing circuit shoWn in
FIG. 2. These are supplied to CPU and regions are extracted
of those Without lack of detail at loW level portion in the SE
image data and those Without lack of detail at high level
portion in the LE image data. Sums ESE, ZLE of pixel data
image data corresponding to tWo frames are treated as one
?le and are temporarily stored to DRAM 11. Further, the
image data corresponding to tWo frames after signal pro
cessing and synthesizing information such as exposure
amount ratio to be used at the time of synthesis are stored as
one ?le to the memory card. The ?le structure includes a
exposure image data 43 as shoWn in FIG. 15. The header
contains information for discriminating betWeen short-time
exposure image data and long-time exposure image data,
exposure amount ratio, etc. In the synthesizing process, the
?le for synthesis is recognized and the header of the ?le is
referred to by a dedicated softWare to automatically generate
and display a Wide dynamic range, synthesized image at
personal computer 17 Which is connected through the per
sonal computer UP 16 to the image pickup apparatus (cam
of the respective extracted regions are obtained. and an
actual exposure amount ratio A' is obtained by the ratio
ZLE/ZSE of the tWo. As shoWn in FIG. 14, the synthesizing
[0115]
process is performed by multiplying the loW exposure image
thus consecutively taken by one release operation at the
era).
Since tWo frames at different exposure amounts are
data SE and the actual exposure amount ratio A'. A discon
image pickup apparatus, an image of moving object such as
tinuity in gradation at the seam betWeen the tWo image data
a person can be taken Without being aWare of taking tWo
is thereby prevented. It should be noted that, in FIG. 14, the
frames. Further, a complicated processing is not performed
Within the image pickup apparatus due to the fact that the
synthesizing process of image data of the taken tWo frames
dotted line represents the case of taking images at a previ
ously set exposure amount ratio A, indicating a discontinuity
in gradation due to variance in the exposure amount ratio.
[0112] A seventh embodiment Will noW be described. The
above embodiments have been shoWn as those in Which
synthesizing process for the generation of Wide dynamic
range, synthesized image is performed Within the image
is performed at the personal computer externally connected
to the image pickup apparatus. Since the processing time is
shorter, it becomes possible to shorten the image taking
interval. Furthermore, since the image data corresponding to
pickup apparatus (camera) by using image data correspond
tWo frames and the synthesizing information thereof are
treated as one ?le and are subjected to synthesis by a
dedicated softWare at the personal computer, a Wide
ing to tWo frames taken at different exposure amounts. The
dynamic range image can be obtained in a similar manner as
synthesizing process, hoWever, is not necessarily required to
be performed Within the image pickup apparatus. It is also
possible to provide a personal computer, printer, etc., With a
operation.
an ordinary taking of image Without performing a special
synthesizing function so as to let it perform synthesis of the
[0116] The synthesizing processing operation to be per
tWo frames.
formed Within the personal computer Will noW be described
brie?y by Way of a ?owchart shoWn in FIG. 16. First, the
header of synthesizing process ?le is referred to and the
[0113] In this embodiment, corresponding to the above
manner, the fundamental construction of the image pickup
apparatus (camera) is similar to the ?rst embodiment shoWn
in FIG. 1. The camera signal processing circuit 6, hoWever,
is different from that shoWn in FIG. 2 in that it does not
require a memory, multiplier, synthesizing circuit, compres
sion circuit, selector, etc., for the performance of synthesiz
ing process and is constructed as a circuit for performing
such ordinary signal processing as y, edge enhancement, etc.
exposure amounts of the respective image data correspond
ing to tWo frames at different exposure amounts are dis
criminated from each other (step S21). The exposure amount
ratio of the tWo image data is computed on the basis of such
exposure amounts and unsuitable portions for synthesis as
those involving motion of the object are detected based on
the tWo image data (step S22). From the result of detection
of the above described unsuitable portions for synthesis, a
Further, DRAM 11 requires a memory corresponding to tWo
frames and the capacity of the memory card becomes a half
the number of images, since data corresponding to tWo
frames must be stored for each one image. It should be noted
that it is also possible to provide detection means for
mined as that synthesis is impossible and the long-time
detecting the remaining capacity of the memory card so that,
When it is detected that the capacity for storing data corre
exposure image data is selected (step S24) and outputted
after performing gradation characteristic transform, etc.,
sponding to tWo frames is not left in the memory card, data
corresponding to one frame can be stored by taking an image
of one frame. It is thereby possible to take image corre
sponding to an ordinary one frame even When the remaining
(step S25).
capacity is less than that for storing image data correspond
ing to tWo frames.
[0114] In the embodiment constructed as the above, tWo
frames of different exposure amounts are consecutively
taken at CCD image pickup device by one release operation
determination is then made as to Whether synthesizing
process is possible or not (step S23). If a motion has been
detected and at the same time if the amount of such motion
goes beyond an alloWable range for correction, it is deter
[0117]
On the other hand, if it is determined as that
synthesizing is possible in processing at step S23 for deter
mining Whether synthesis is possible or not, it is then
determined With respect to reference pixels Whether the
reference pixels are of portions unsuitable for synthesis or
not (step S26). If not of portions unsuitable for synthesis, the
synthesizing process is performed Without alteration (step
S27). On the other hand, if of portions unsuitable for
US 2007/0146538 A1
Jun. 28, 2007
synthesis, the portions unsuitable for synthesis are corrected
using for example the method shoWn in FIG. 4 (step S28).
Then, after White balance adjustment (step S29), the gener
suitable at the time of backlighted condition, a synthesized
image can be obtained as suitable for the backlighted
condition.
ated, synthesized image is subjected to processing for
dynamic range compression and is outputted (step S30).
[0121] Further, in accordance With an eighth aspect of the
invention, since ON/OFF sWitching of the generation pro
cessing operation at the means for generating Wide dynamic
[0118]
While the above embodiment has been shoWn as
that for letting an externally connected personal computer
perform synthesizing process, it is also possible that a
synthesizing section be similarly provided at the printer 19
Which is connected from the outside through the printer UP
range, synthesized image is thus controlled based on infor
mation set in the image pickup apparatus before the taking
of image, images can be readily taken in a manner suitable
synthesizing section to perform the image synthesizing
for the generation of Wide dynamic range, synthesized
image Without requiring detection of the conditions for the
generation of synthesized image from taken images. In
process and to output a synthesized image. In such case,
accordance With the ninth aspect of the invention, ON/OFF
18 so that the header of the ?le is referred to at such
direct printing becomes possible.
[0119] As has been described by Way of embodiments, in
accordance With the ?rst aspect of the present invention,
since ON/OFF sWitching control of the generation process
ing operation at the means for generating Wide dynamic
range, synthesized image is performed based on image
signals corresponding to a plurality of previously taken
frames, an image pickup apparatus can be achieved as
sWitching of the generation processing operation of synthe
sized image can be performed in a manner linked to the
shutter speed and zoom information so that the generation
processing of synthesized image can be turned OFF to
prevent a failed, synthesized image from being outputted
When the shutter speed is sloW or When the zoom ratio is
high. In accordance With the tenth aspect of the invention,
since means for correcting exposure amounts of image
signals and ON/OFF sWitching control means based on the
image taking mode of the exposure amount correction
capable of accurately performing ON/OFF sWitching control
of processing operation for the generation of Wide dynamic
means are provided, exposure amount is not alWays detected
range, synthesized image. In accordance With the second
and the exposure amounts can be corrected to obtain a
aspect of the invention, since motion detection means is
provided so as not to perform the generation processing of
synthesized image When a motion of the object is detected,
it is possible When the object has been moved to prevent a
failed, synthesized image from being outputted. In accor
dance With the third aspect of the invention, since the motion
detection means and the motion amount determination
means and motion correction means are provided so as to
perform synthesizing process With correcting the amount of
motion When the detected motion amount is Within the
alloWable range for correction, it is possible to provide a
grater range for the conditions of object of Which a Wide
dynamic range, synthesized image can be formed.
[0120] Further, in accordance With the fourth aspect of the
invention, since ON/OFF sWitching of the generation pro
cessing operation of Wide dynamic range, synthesized image
is performed on the basis of object information obtained
before the taking of image, images can be taken in a manner
suitable synthesized image only When there is a possibility
of shift from the set exposure amounts of the image signals
corresponding to a plurality of frames of different exposure
amounts. In accordance With the eleventh aspect of the
invention, since the exposure amount correction means is
caused to operate at the time of a strobe image taking mode,
a suitable synthesized image for example Without disconti
nuity in gradation can be generated even if variance occurs
in the amount of strobe emission at the time of strobe image
taking mode.
[0122] Further, in accordance With the tWelfth aspect of
the invention, synthesizing process of the synthesized image
is performed at an external synthesizing apparatus, it
becomes unnecessary to perform a complicated synthesizing
process Within the image pickup apparatus so that time for
processing of image taking is shortened and it becomes
possible to reduce the image taking interval. In accordance
With the thirteenth aspect of the invention, since means for
suitable for the generation of Wide dynamic range, synthe
detecting the remaining capacity of recording medium is
sized image. In accordance With the ?fth aspect of the
provided and image signals corresponding to one frame is
recorded When the remaining capacity is less than that for
recording image signals corresponding to a plurality of
frames, an ordinary image taking corresponding to one
invention, since images are taken in a forced strobe emission
mode and the generation processing of synthesized image is
performed When a motion in the object has been detected at
the motion detection means, the motion in the object can in
effect be stopped even of an object involving a motion,
frame can be performed even When it is impossible to obtain
a synthesized image. In accordance With the fourteenth
aspect of the invention, since image signals corresponding to
making it possible to readily generate a Wide dynamic range,
synthesized image. In accordance With the sixth aspect of the
a plurality of frames and the synthesizing information
invention, since the generation processing of synthesized
thereof are recorded as a single ?le to the recording medium,
image is performed by using a higher shutter speed When a
motion of the object has been detected, it is possible to
lessen motion of the object in image signals corresponding
them similarly as in the case of image signals corresponding
to an ordinary single frame. In accordance With the ?fteenth
it is possible to perform synthesizing process by treating
to a plurality of frames. Further, since motion of the object
aspect of the invention, since image signals corresponding to
can be in effect reduced in extent even for an object
a plurality of frames and synthesizing information thereof
involving motion by using a higher shutter speed, it is
possible to readily generate a Wide dynamic range, synthe
sized image. In accordance With the seventh aspect of the
are recorded as a single ?le in a recording medium from
Which said ?le is read out to effect synthesizing process, a
Wide dynamic range synthesized image can be outputted
similarly as in the case of an ordinary taken image Without
invention, since a backlighted condition is detected at an AB
detection circuit so as to perform the generation processing
a special operation. In accordance With the sixteenth aspect
of Wide dynamic range, synthesized image in a manner
of the invention, a recording medium can be provided as that
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement