US008416317B2 (12) Unlted States Patent (10) Patent No.: Palum (54) (75) US 8,416,317 B2 (45) Date of Patent: Apr. 9, 2013 AUTOMATIC FOCUS SYSTEM 6,637,896 B2 CALIBRATION FOR IMAGE CAPTURE 6,796,655 B2 9/2004 Sekl SYSTEMS 6,854,474 B1 2/2005 Inventor: Chen _ _ 6,984,039 B2 1/2006 Ag0st1nell1 7,961,973 B2 * 6/2011 Russell J. Palum, Rochester, NY (US) H .......................... .. 382/274 - ung (Cont1nued) (73) Assignee: Apple Inc., Cupertino, CA (US) ( * ) Notice: 10/2003 Li et a1. FOREIGN PATENT DOCUMENTS 1’??? Subject115522313330? to any disclaimer, digsusted the term under of this 35 W0 W0 03/038517 é 5/2003 OTHER PUBLICATIONS (21) Appl' NO': 12/503’095 (22) Filed: Jul- 15, 2009 (65) Light Blue Optics, Ltd., “PVProTM Enabling Personal Video Projec tors ,WWW.l1ghtblue0pt1cs.c0m. (Continued) Prior Publication Data Us 2009/0273680 A1 Nov. 5, 2009 pr’rf’ary Exam?” * Aung S Moe Asszslanl Exammer * Mekonnen DagneW Related US. Application Data (63) gtzqzoorgng’lg?igceféiegr Continuation of application No. 11/496,913, ?led on Aug. 1, 2006, noW Pat. No. 7,609,958. (51) Int Cl H04N ‘9/73 H04N 5/76 (2006 01) (200601) G03B 13/00 (200601) ' _ (57) * Wong’ Cabeno’ Lutsch’ ABSTRACT Imaging systems and methods for calibrating imaging sys P rov1‘de ‘1 . The 1ma ' g’1ng s y stem has a b o dy, a scene tems are image capture system that captures images using a taking lens system that can be set to a plurality of different focus dis / _ / (52) (58) US. Cl. ...... .... ...... 348/2231, 348 345, 348 229.1 FIeld of ClassI?catIon Search 348/223.1*225.1, (56) See apphcanon ?le for Complete Search hlstory' References Cited _ 348/255’ 229'1’ 20812’ 345’ 36_5’366’ 361 U.S. PATENT DOCUMENTS tances, and a range?nder that is capable of determining a distance between the imaging system and at least one portion of a ?eld of View of the talking lens system_ The method comprises: automatically capturing a ?rst calibration image of a ?rst ?eld of vieW through the taking lens system With the taking lens system set to a ?rst focus distance setting; identi fying a portion of the ?rst calibration image having a prede termined degree of focus; using the range?nder to determine a ?rst calibration distance from the imaging device to the 2 gan?tet 11' l identi?ed portion. A focus correlation is determined based 5’440’369 A 8/1995 Tglgataaei j ' upon the ?rst calibration distance and the ?rst focus distance 5,715,483 A 5,877,809 A 2/1998 Omata et a1. 3/1999 Omata et a1. Semng 6,067,114 A 5/2000 Omata et a1. 7 Claims, 13 Drawing Sheets 34 27 747 76 AUDIO 22\ 23 25 DRIVER IMA GE SENSOR CIRCUITRY a2 CONTROLLER 26 \ SIGNAL 54 MEMORY COMMUNICATION IN TERFA CE MODULE 24 REMO VA BLE MEMORY 52 REMOTE MEMORY US 8,416,317 B2 Page 2 U.S. PATENT DOCUMENTS 2002/0118151 2002/0159101 2002/0176015 2003/0081184 2005/0024606 2005/0168705 2006/0061676 A1 8/2002 Chen A1 * 10/2002 Alderson et a1. ............ .. 358/504 A1 A1 A1 A1 A1 11/2002 5/2003 2/2005 8/2005 3/2006 Lichtfuss Li et al. Li et al. Li et al. Suda OTHER PUBLICATIONS Jacqueline Hewett, “Tiny Image Projectors Take a Step Towards the Shops”, Jul./Aug. 2003, www.optics.org. Anne Eisenberg, “What’s Next; For Your Viewing Pleasure, a Pro jector in Your Pocket”, Nov. 4, 2004, www.nytimes.com. * cited by examiner US. Patent Apr. 9, 2013 Sheet 2 0f 13 US 8,416,317 B2 US. Patent Apr. 9, 2013 Sheet 3 0f 13 US 8,416,317 B2 @ 80 *1 ACTIVATE CAMERA 32% CAPTURE CALIBRATION IMAGE I I IDENTIFYA PORTION OF THE 34% CALIBRATION IMAGE THAT HAS A PREFERRED LEVEL OF FOCUS I USE RANGEFINDER TO MEASURE CALIBRATION 35% DISTANCE FROM THE IMAGE CAPTURE SYSTEM TO THE IDENTIFIED PORTION I DETERMINE A FOCUS CORRELATION DA TA SET 88 ,h BASED UPON CALIBRATION DISTANCE AND THE FOCUS DISTANCE SETTING USED TO CAPTURE THE FIRST IMAGE I STORE INDICATION OF 90 ’\/ DETERMINED FOCUS CORRELATION 94 x USE RANGEFINDER TO DE TEC T CAPTURE DISTANCE I 96“ CAPTURE IMAGE USING INDICATED FOCUS 98 YES CORRELATION R mg’?és AND CAPTURE DISTANCE TO DETERMINE FOCUS SETTING ? FIG. 4 US. Patent Apr. 9, 2013 Sheet 4 0f 13 US 8,416,317 B2 M3 M £2 Q 1.1.6 m /\ M FIG. 5 US. Patent Apr. 9, 2013 Sheet 6 0f 13 US 8,416,317 B2 68E 120 ’\/ ACTIVATE CAMERA 122 GO TO STEP 82 124 A, USE THROUGH-THE-LENS FOCUS TO SET CAMERA FOCUS POSITION FOR AN IDENTIFIED PORTION OF THE IMAGE I 126 “1 CAPTURE ARCHIVAL IMAGE I USE RANGEFINDER TO MEASURE DISTANCE FROM 128 “A THE IMAGE CAPTURE SYSTEM To THE IDENTIFIED PORTION OF THE IMAGE I DETERMINE A FOCUS CORRELATION DATA SET BASED 130 AV UPON CALIBRATION DISTANCE AND THE FOCUS DISTANCE SETTING USED TO CAPTURE THE ARCHIVAL IMAGE I STORE INDI CA TI ON OF 132 ’\_» DETERMINED FOCUS CORRELA TION 134 136 ,\I USE RANGEFINDER TO DE TEC T CAPTURE DISTANCE * CAPTURE IMAGE USING INDICA TED FOCUS ‘'38 “P 140 YES CORRELATION $195515 AND CAPTURE DISTANCE TO 9 DETERMINE FOCUS SETTING ' FIG. 9 US. Patent Apr. 9, 2013 US 8,416,317 B2 Sheet 7 0f 13 (START) 1 80m ACTIVATE CAMERA I 82 CAPTURE CALIBRATION IMAGE I IDENTIFYA PORTION OF THE 84% CALIBRATION IMAGE THAT HAS A PREFERRED LEVEL OF FOCUS I USE RANGEFINDER TO MEASURE CALIBRATION 86x DISTANCE FROM THE IMAGE CAPTURE SYSTEM TO THE IDENTIFIED PORTION I DETERMINEA FOCUS CORRELA TION DA TA SET 88 "w BASED UPON CALIBRATION DISTANCE AND THE FOCUS DISTANCE SETTING USED TO CAPTURE THE FIRST IMAGE STORE INDI CA TI ON OF 90% 98 DETERMINED FOCUS C ORRELA TI ON MORE PICTURES ? YES 92 CONDITION ETECTED 9 97 YES SUBJECT PORTION OF IMA GE IN FOCUS DE TEC T OPERA TING CONDITIONS 9 93 YES OPERATING CONDITIONS 96x RANGEFINDER TO 94x DEUSE TECT CAPTURE DISTANCE FIG. 10 CAPTURE IMAGE USING INDICATED FOCUS CORRELATION AND CAPTURE DISTANCE TO DETERMINE FOCUS SETTING US. Patent 150 ’\J Apr. 9, 2013 Sheet 8 0f 13 US 8,416,317 B2 @ ACTIVATE CAMERA GO TO STEP 82 USE RANGEFINDER TO MEASURE DISTANCE FROM 154 ’\J THE IMAGE CAPTURE SYSTEM TO THE SUBJECT AREA PORTION I 156 Av 758 ' \- USE PRE-PROGRAMMED FOCUS CORRELATION TO DETERMINE FOCUS DISTANCE I I DETERMINE A FOCUS CAPTURE ARCHIVAL IMAGE CORRELATION DATA SET BASED 160 A“ UPON CALIBRATION DISTANCE AND THE FOCUS DISTANCE SETTING USED TO CAPTURE THE ARCHIVAL IMAGE I STORE INDICA TION OF 162 m DETERMINED FOCUS C ORRELA TI ON CONDITION I ETECTED 166 ,\_I USE RANGEFINDER TO DE TEC T CAPTURE DISTANCE i CAPTURE IMAGE USING INDICA TED FOCUS 168 “P CORRELATION AND CAPTURE DISTANCE TO DETERMINE FOCUS SETTING 1 70 YE S MORE PI C TgJRES FIG. 11 US. Patent Apr. 9, 2013 US 8,416,317 B2 Sheet 9 0f 13 @ .1 80 A C TI VATE CAMERA 182 CAPTUR CONDITION E TECTED GO TO STEP 82 184m DETECT OPERATING CONDITIOMS) 186% SELECT FOCUS CORRELATION BASED ON OPERATING CONDITIONS I USE RANGEFINDER TO MEASURE DISTANCE FROM THE IMAGE CAPTURE SYSTEM TO THE SUBJECTAREA PORTION 190'» CAPTURE ARCHIVAL IMAGE USING RANGEFINDING DIS TANCE AND DETERMINED FOCUS CORRELATION i USE THROUGH-THE-LENS FOCUS 192% TO DETERMINE SETTINGS FOR CAPTURING A CALIBRATION IMAGE i DETERMINE A FOCUS CORRELATION DATA SET BASED UPON CALIBRATION 194'» DISTANCE AND THE FOCUS DISTANCE SETTING USED TO CAPTURE THE ARCHIVAL IMAGE i STORE INDICATION OF 196 ’\I DETERMINED FOCUS CORRELATION 198 CAPTUR CONDITION E TEC TED 200 '\, USE RANGEFINDER TO DE TEC T CAPTURE DISTANCE I CAPTURE IMAGE USING INDICATED FOCUS CORRELATION AND CAPTURE DISTANCE TO DETERMINE FOCUS SETTING 204 YES MORE PICTURES ? FIG. 12 US. Patent Apr. 9, 2013 Sheet 11 0113 US 8,416,317 B2 START DETERMINING FOCUS CORRELATION STEP DOES MEASURED 212 7 SELECT FITTING FOCUS CORRELATION 214 7 EVALUA TE CALIBRATION IMAGE TO IDENTIFY EA CH AREA HAVING _"'_'_"* A FOCUS THATAT OR ABOvEA PREDETERMINED FOCUS LEVEL 224 / CAPTURE ADDITIONAL CALIBRATION IMAGE 222 7 I MOVE LENS SYSTEMA TAKING LENS POSITION NOT PREVIOUSLY UsED TO CAPTUREA CALIBRATION IMAGE+- 216 7 l MEASURE DISTANCE TO EACH IDENTIFIED AREA USING RA NGEFINDING DATA 218 7DETERMINEI RANGE OF DISTANCES AND ASSOCIATE RANGE OF DIsTANCEs WITH LENS SETTINGS UsED TO CAPTURE CALIBRATION IMAGE. DETERMINE FOCUS CORRELATION USING MEASURED DISTANCES AND FOCUS DISTANCE SETTINGS I END DETERMINING FOCUS CORRELATION STEP FIG. 16 US. Patent Apr. 9, 2013 Sheet 13 0f 13 310% PROJECT IMAGE I 312% CA PTURE IMA GE OF PROJE C TED IMA GE I 314% CALCULATE DIFFERENCE MAP I 316% STORE DIFFERENCE MAP FOR USE I 318% DETERMINE RANGEFINDING DISTANCES I DE TECT VARIATIONS IN RANGEFINDING DISTANCES I DETERMINE IMAGE 322 ’\J ADJUSTMENTS BASED UPON RANGEFINDING DISTANCES END FIG. 18 US 8,416,317 B2 US 8,416,317 B2 1 2 AUTOMATIC FOCUS SYSTEM CALIBRATION FOR IMAGE CAPTURE SYSTEMS tured through tWo lens systems that are separated laterally by a distance and determine the distance to the scene through triangulation. The focus setting of the adjustable focus lens system is then determined using a calibrated preprogrammed curve or look-up table that correlates scene distances With CROSS-REFERENCE TO RELATED APPLICATIONS lens positions that can be used to capture objects at the scene distance in focus. A Wide variety of range?nder type autofo cus systems are very Well knoWn in the art. This Application is a continuation of US. Ser. No. 11/496, 913, ?led Aug. 1, 2006, now US. Pat. No. 7,609,958 issued Oct. 27, 2009. Range?nder type autofocus systems have the advantage of being very fast With some having a response time that can be in the range of 0.01 -0.05 second. HoWever, the focus quality produced by some range?nder type autofocus systems can vary When they are used in different operating conditions. For example, temperature and humidity can affect the calibration This application is co-?led With and has related subject matter to US. Patent Application Publication No. 20090273681 published Nov. 5, 2009. of the distance to focus lens position curve due to ?uctuations FIELD OF THE INVENTION in the refractive index and dimensions of both the range?nder autofocus system components and the taking lens compo The invention relates to automatic calibration of imaging systems to improve image quality and enable faster operation. BACKGROUND OF THE INVENTION nents. 20 With the lens system positioned at a plurality of different focus distances. For example, in a contrast based “through the-lens” autofocus system a plurality of different images Imaging systems are limited in terms of image quality by artifacts introduced by the environment that they are operated in. One Way to avoid capturing or creating images that have such artifacts is to calibrate the imaging system in an envi The “through-the-lens” autofocus system determines focus settings using analysis of a series of images captured (e.g. 5-20) are captured With the taking lens in different focus 25 lens positions in a so-called hill climb method. The contrast ronment that is similar to the environment in Which it Will be present in the captured images is compared and the image used. Often this is done When the imaging device is manufac With the greatest contrast is determined to be the image With the best focus conditions (often the best focus lens position is further re?ned by interpolating the contrast values betWeen tured. For example, it is knoWn in the art to use test ?xtures to calibrate autofocus systems in ?lm cameras While such cam eras are Within an operating range of environmental condi 30 images). The “through-the-lens” type autofocus system is tions. very accurate since it measures focus quality directly from In particular, one aspect of an imaging system that bene?ts from calibration is the autofocus system in an imaging sys images captured With the high quality talking lens. tem. Many ?lm cameras, digital cameras and scanners cap systems can be relatively sloW in determining a focus setting. ture images using an imager and a lens system With an adjust able focus lens system. Typically, the focus distance of such an adjustable focus lens system can automatically be set to one of a plurality of different settings by sensing, control and drive systems that are adapted to provide optimal focus of HoWever, conventional “through-the-lens” type autofocus 35 For example, such systems can take as long as 0.5-2.0 seconds to determine a focus distance. This is because such “through the-lens” autofocus systems require the capture and process ing of a number of images. Accordingly, in some digital cameras, the tWo types of What is determined to be a subject area in a scene. Lens 40 autofocus systems are used together in a hybrid system in systems that have automatically adjustable focus settings are Which the range?nder type autofocus system is used to pro referred to herein as autofocus systems. vide a fast estimation of a focus setting that is then folloWed It Will be appreciated that it is important to properly cali brate such autofocus systems. In the above example, focus settings for ?lm cameras are calibrated by using the test ?xture to monitor an image provided by the lens system of by the use of the “Through-the-lens” autofocus system to re?ne the focus setting. For example, US. Pat. No. 6,864,474 entitled “Focusing Apparatus forAdjusting Focus of an Opti cal Instrument”, ?led by MisaWa on Jan. 10, 2003, describes 45 such a ?lm camera and adjusting the lens system until the lens system reaches a ?rst setting Where a test target located at a the coordinated use of a range?nder type autofocus system With a through-the-lens autofocus system. In MisaWa, the ?rst distance from the camera is in focus. The range?nder for focus position of the taking lens is determined by both the range?nder based autofocus system and the “through-the the ?lm camera is then used to measure the distance to the test 50 target and thereafter the range?nder Will position the lens lens” autofocus system, the difference betWeen the focus system at the ?rst setting Whenever the range?nder measures that distance. This process is then repeated for a plurality of position determined by the range?nder type autofocus system and the focus position determined by the “through-the-lens” other test targets, each located at one of a range of additional distances so that the range?nding measurements are associ type autofocus system is stored for future reference. In sub 55 sequent image capture episodes, the stored difference infor ated With each of a plurality or lens focus settings. Digital cameras typically use one of tWo types of autofocus mation is used to re?ne the number of images captured and systems: range?nder type autofocus systems or a “through the-lens” type autofocus system to automatically determine the hill climb method to determine the focus lens position taking lens focus settings. A range?nder autofocus system analyZed by the “through-the-lens” type autofocus system in With best focus, thereby reducing the number of images cap 60 tured and processed When the range?nder has been accurate uses sensors such as optical range?nders or sonic range?nd and increasing the number of images captured and processed ers to determine a distance from a camera to one or more When the range?nder has been inaccurate. HoWever, the method described by MisaWa assumes that the performance of the range?nder, adjustable focus lens system, and control portions of a scene Within a ?eld of vieW of the adjustable lens system. Common range?nder type autofocus systems include active and passive systems. In one example of an active range?nder type system, the range?nder type autofocus sys system are consistent over time, do not ?uctuate With varia tions in environmental conditions and do not otherWise tem compares tWo loW-resolution images that have been cap change or drift over time. 65 US 8,416,317 B2 4 3 MisaWa also does not eliminate the use of multiple image FIG. 4 shoWs a block ?oW diagram of one embodiment of capture and processing used by the “through-the-lens” type an auto-calibrating range?nder-based autofocus system; autofocus system so that the hybrid autofocus as described by MisaWa remains sloW. A further aspect of an imaging system that Would bene?t from calibration is a projection system in order to ensure that a projection lens system is properly focused. There have been efforts to provide automatic feed back systems to this end. For example, US. Patent Applica tion Publications US2005/0168705 and US2005/0024606 FIG. FIG. FIG. FIG. FIG. 5 illustrates one example of a calibration image; 6 illustrates one example of a focus correlation; 7 illustrates one example of a focus correlation; 8 illustrates one example of a focus correlation; 9 shoWs another embodiment of a method for cali brating an imaging device; FIG. 10 shoWs another embodiment of a method for cali both by Li et al., describe projection systems Which include feedback of a projected image by an imaging sensor system. brating an imaging device; In this case Li et al. teaches the use of the imaging sensor brating an imaging device; FIG. 11 shoWs another embodiment of a method for cali FIG. 12 shoWs another embodiment of a method for cali system to aid in focusing the projector. Li et al. also teaches brating an imaging device; the use of an imaging sensor system to enable the projection FIG. FIG. FIG. FIG. system to correct for projector-to-surface orientation prob lems, such as correcting to adjust for keystone in the projected image, or to ?t the projected image Within the edge of a projection screen. Thus, Li et al. discloses, essentially, a “Through-the-lens” focus system With orientation compen sation. HoWever, here again calibration of such a system is 13 shoWs one example of a focus correlation; 14 shoWs one example of a focus correlation; 15 shoWs one example of a focus correlation; 16 shoWs another embodiment of a method for deter mining a focus correlation; 20 FIG. 17 shoWs an embodiment of an imaging device With an associated projection system; and FIG. 18 shoWs a method for calibrating the projection typically performed only during manufacturing or during a manual service procedure. Therefore the need persists to improve imaging systems system. through neW calibration approaches. 25 DETAILED DESCRIPTION OF THE INVENTION SUMMARY OF THE INVENTION Before explaining the present embodiments in detail, it is Image capture systems and methods for calibrating an imaging system are provided. In one aspect of the invention, the imaging system has a body, a scene image capture system that captures images using a taking lens system that can be set to a plurality of different focus distances, and a range?nder that is capable of determining a distance betWeen the imaging to be understood that the embodiments are not limited to the particular descriptions and that it can be practiced or carried 30 system and at least one portion of a ?eld of vieW of the taking lens system. The method comprises the steps of: automati cally capturing a ?rst calibration image of a ?rst ?eld of vieW through the taking lens system With the taking lens system set 35 image capture system 22 and autofocus system 27 are mounted. Scene image capture system 22 having a taking lens ?rst calibration image having a predetermined degree of system 23, a scene image sensor 24, a signal processor 26, an 40 system using the range?nder, and setting the focus distance setting for the taking lens system for the capture of the archi val image based upon the determined focus correlation and the determined capture distance. have one or more elements. 45 50 stationary element or elements (not shoWn) by lens driver 25 that is motoriZed for automatic movement. Lens driver 25 55 controls both the lens focal length and the lens focus position of taking lens system 23 and sets a lens focal length and/or position based upon signals from signal processor 26, an optional automatic range ?nder system 27, and/or controller 32. It Will be appreciated that in other embodiments, taking lens system 23 can comprise lenses having shapes that can be changed to provide an in situ variation of lens focus distance by modi?cation of lens characteristics such as curvature. 60 FIG. 1 shoWs a block diagram of one embodiment of an Examples of lens systems of this type include those that use, for example, liquid lens technology knoWn to those of skill in the art. In such embodiments, lens driver 25 can provide signals to cause focus distance changes in the lens Without use of a motor. Examples of such liquid lenses include lenses sold image capture system; FIG. 2 shoWs a back, elevation vieW of the image capture system of FIG. 1; FIG. 3 shoWs a front, elevation vieW of the image capture system of FIG. 1; Taking lens system 23 is of an automatically adjustable type. In the embodiment shoWn in FIGS. 1-3, taking lens system 23 is automatically adjusted to a variety of focus settings. In the example embodiment shoWn in FIG. 1, taking lens system 23 is a 6x Zoom lens unit in Which a mobile element or elements (not shoWn) are driven, relative to a BRIEF DESCRIPTION OF THE DRAWINGS In the detailed description of the preferred embodiments presented beloW, reference is made to the accompanying draWings, in Which: optional display driver 28 and a display 30. In operation, light from a scene is focused by taking lens system 23 to form an image on scene image sensor 24. Taking lens system 23 can the scene image capture system is to be used to capture an archival image of a scene and, in response thereto, performing the steps of: determining a capture distance from the imaging system to a portion of the ?eld of vieW of the taking lens imaging system 10. FIG. 2 shoWs a back, elevation vieW of the imaging system 10 of FIG. 1, While FIG. 3 shoWs a front elevation vieW of the imaging system 10 of FIG. 1. As is shoWn in FIGS. 1-3, imaging system 10 takes the form of a digital camera 12 comprising a body 20 to Which a scene to a ?rst focus distance setting; identifying a portion of the focus; using the range?nder to determine a ?rst calibration distance from the imaging device to the identi?ed portion of the ?rst calibration image; determining a focus correlation based upon the ?rst calibration distance and the ?rst focus distance setting, said focus correlation associating different range?nder determined distances With each of the plurality of focus distance settings With at least one range?nder deter mined distance; detecting a capture condition indicating that out in various Ways. FIG. 1 shoWs a block diagram of an embodiment of an 65 commercially under the name of Fluidfocus by Royal Philips Electronics, Amsterdam, The Netherlands and other liquid lens products sold by PGS Precision, Singapore. US 8,416,317 B2 5 6 In the embodiment of FIG. 1, the focus position of taking lens system 23 can be automatically selected by a variety of knoWn strategies. For example, in one embodiment, scene image sensor 24 is used to provide multi-spot autofocus using What is called the “Through focus” or “Whole Way scanning” approach. As described in commonly assigned US. Pat. No. 5,877,809 entitled “Method Of Automatic Object Detection In An Image”, ?led by Omata et al. on Oct. 15, 1996, the disclosure of Which is herein incorporated by reference. If the Signal processor 26 receives image signals from scene image sensor 24 and transforms the image signals into a digital image in the form of digital data. The image can comprise one or more still images, multiple still images and/ or a stream of apparently moving images such as a video segment. Where the digital image data comprises a stream of apparently moving images, the digital image data can com prise image data stored in an interleaved or interlaced image form, a sequence of still images, and/ or other forms knoWn to those of skill in the art of digital video. target object is moving, object tracking may be performed, as described in commonly assigned US. Pat. No. 6,067,114 Signal processor 26 can apply various image processing algorithms to the image signals When forming a digital image. entitled “Detecting Compositional Change in Image” ?led by These can include but are not limited to color and exposure Omata et al. on Oct. 26, 1996, the disclosure of Which is herein incorporated by reference. In an alternative embodi balancing, interpolation and compression. Where the image signals are in the form of analog signals, signal processor 26 also converts these analog signals into a digital form. In certain embodiments, signal processor 26 can be adapted to ment, the focus values determined by “Whole Way scanning” are used to set a rough focus position, Which is re?ned using a ?ne focus mode, as described in commonly assigned US. process image signals so that the digital image formed Pat. No. 5,715,483, entitled “Automatic Focusing Apparatus thereby appears to have been captured at a different Zoom and Method”, ?led by Omata et al. on Oct. 11, 1998, the disclosure of Which is herein incorporated by reference. 20 Digital camera 12 has a range?nder 27. Range?nder 27 is adapted to determine a distance from imaging system 20 to at least a portion of a ?eld of vieW of taking lens system 23. Range?nder 27 can determine the distance as an absolute distance measurement, such as a measurement in feet or meters, or as a relative distance measurement to the focus scene image sensor 24 and interpolating the subset of the image signals to form the digital image. This is knoWn gen erally in the art as “digital Zoom”. Such digital Zoom can be 25 used to provide electronically controllable Zoom adjusted in ?xed focus, manual focus, and even automatically adjustable focus systems. Controller 32 controls the operation of the imaging system 30 scene image capture device 22, display 30 and memory such setting for taking lens system 23 that is appropriate for the distance to the subject Without use of it. Range?nder 27 can operate lens driver 25, directly or as shoWn in FIG. 1, can 10 during imaging operations including, but not limited to, provide signals to signal processor 26 or controller 32 from Which signal processor 26 or controller 32 can generate sig nals that are to be used for image capture. A Wide variety of suitable multiple sensor range?nders 27 knoWn to those of skill in the art are suitable for use. For example, US. Pat. No. as memory 40. Controller 32 causes scene image sensor 24, signal processor 26, display 30 and memory 40 to capture, present and store scene images in response to signals received from a user input system 34, data from signal processor 26 35 5,440,369 entitled “Compact Camera With Automatic Focal al. on Nov. 30, 1993, the disclosure of Which is herein incor 40 imaging system 10. 45 Controller 32 cooperates With a user input system 34 to alloW imaging system 10 to interact With a user. User input system 34 can comprise any form of transducer or other device capable of receiving an input from a user and convert ing this input into a form that can be used by controller 32 in the scene is divided into a grid of areas or spots, and the optimum focus distance is determined for each spot. One of the spots is identi?ed as the subject of the image and the focus distance for that spot is used to set the focus of taking lens system 23. A feedback loop is established betWeen lens driver 25 and controller 32 and/or range?nder 27 so that the focus position of taking lens system 23 can be rapidly set. operating imaging system 10. For example, user input system 34 can comprise a touch screen input, a touch pad input, a 4-Way sWitch, a 6-Way sWitch, an 8-Way sWitch, a stylus 50 Taking lens system 23 is also optionally adjustable to pro vide a variable Zoom. In the embodiment shoWn lens driver 25 automatically adjusts the position of one or more mobile elements (not shoWn) relative to one or more stationary ele ments (not shoWn) of taking lens system 23 based upon sig nals from signal processor 26, an automatic range?nder 27, 55 and/ or controller 32 to provide a Zoom magni?cation. Taking lens system 23 can be of a ?xed Zoom setting, manually system, a trackball system, a joystick system, a voice recog nition system, a gesture recognition system or other such systems. In the digital camera 12 embodiment of imaging system 10 shoWn in FIGS. 1 and 2 user input system 34 includes a capture button 60 that sends a capture signal to controller 32 indicating a desire to capture an archival image. User input system 34 can also include other buttons including the joystick 66, mode button 67, and the select it button 68 shoWn in FIG. 2. Sensors 36 are optional and can include light sensors and other sensors knoWn in the art that can be used to detect adjustable and/or can employ other knoWn arrangements for providing an adjustable Zoom. and data received from optional sensors 36. Controller 32 can comprise a microprocessor such as a programmable general purpose microprocessor, a dedicated micro-processor or micro-controller, a combination of discrete components or any other system that can be used to control operation of Length Dependent Exposure Adjustments” ?led by Tabata et porated by reference, discloses one such range?nder 27. The focus determination provided by range?nder 27 can be of the single-spot or multi-spot type. Preferably, the focus determi nation uses multiple spots. In multi-spot focus determination, setting than that actually provided by the optical lens system. This can be done by using a subset of the image signals from 60 conditions in the environment surrounding imaging system Light from the scene that is focused by taking lens system 10 and to convert this information into a form that can be used 23 onto scene image sensor 24 is converted into image signals representing an image of the scene. Scene image sensor 24 by controller 32 in governing operation of imaging system 10. Sensors 36 can include audio sensors adapted to capture sounds. Such audio sensors can be of conventional design or can comprise a charge couple device (CCD), a complimen tary metal oxide sensor (CMOS), or any other electronic image sensor knoWn to those of ordinary skill in the art. The image signals can be in digital or analog form. 65 can be capable of providing controllably focused audio cap ture such as the audio Zoom system described in US. Pat. No. 4,862,278, entitled “Video Camera Microphone With Zoom US 8,416,317 B2 7 8 Variable Acoustic Focus”, ?led by Dann et al. on Oct. 14, 1986. Sensors 36 can also include biometric sensors adapted to detect characteristics of a user for security and affective as the MPEG-4 (Motion Pictures Export Group) or Apple QuickTimeTM standard can be used to store digital image data in a video form. Other image compression and storage forms imaging purposes. Sensors 36 can also include temperature can be used. sensors or humidity sensors to detect the environmental con The digital images and metadata can be stored in a memory ditions surrounding the imaging system 10. Where a need for such as memory 40. Memory 40 can include conventional additional scene illumination is determined, controller 32 can memory devices including solid state, magnetic, optical or cause an optional source of arti?cial illumination 37 such as other data storage devices. Memory 40 can be ?xed Within imaging system 10 or it can be removable. In the embodiment of FIG. 1, imaging system 10 is shoWn having a memory card a light, strobe, or ?ash system to emit light. Controller 32 causes an image signal and corresponding digital image to be formed When a capture condition is slot 46 that holds a removable memory 48 such as a remov detected indicating a desire of a user to capture an archival able memory card and has a removable memory interface 50 image. Typically, the capture condition occurs When a user for communicating With removable memory 48. The digital depresses capture button 60, hoWever, controller 32 can deter images and metadata can also be stored in a remote memory system 52 that is external to imaging system 10 such as a mine that a capture condition exists at a particular time, or at a particular time after capture button 60 is depressed. Alter personal computer, computer netWork or other imaging sys natively, controller 32 can determine that a capture condition exists When optional sensors 36 detect certain environmental tem. In the embodiment shoWn in FIGS. 1 and 2, imaging sys conditions, such as optical or radio frequency signals. Fur ther, controller 32 can determine that a capture condition exists based upon affective signals obtained from sensors 36 20 that are adapted to sense the physiology of a user. Controller 32 can also be used to generate metadata in association With each image. Metadata is data that is related to a digital image or a portion of a digital image but that is not example, an optical, radio frequency or other Wireless circuit 25 necessarily observable in the image itself. In this regard, controller 32 can receive signals from signal processor 26, camera user input system 34 and other sensors 36 and, option ally, generate metadata based upon such signals. The meta data can include, but is not limited to, information such as the device. Controller 32 can also receive information and instructions from signals received by communication module type of scene image sensor 24, mode setting information, 54 including but not limited to, signals from a remote control device (not shoWn) such as a remote trigger button (not integration time information, taking lens system 23 setting information that characterizes the process used to capture the imaging system 10 to form the scene image. The metadata can also include but is not limited to any other information deter mined by controller 32 or stored in any memory in imaging system 10 such as information that identi?es imaging system 10, and/ or instructions for rendering or otherWise processing or transducer that converts image and other data into a form, such as an optical signal, radio frequency signal or other form of signal, that can be conveyed to an external device. Com munication module 54 can also be used to receive a digital image and other information from a host computer, netWork (not shoWn), or other digital image capture or image storage 30 time, date and location that the scene image Was captured, the scene image and processes, methods and algorithms used by tem 10 has a communication module 54 for communicating With external devices such as, for example, remote memory system 52. The communication module 54 can be for 35 shoWn) and can operate imaging system 10 in accordance With such signals. Signal processor 26 and/or controller 32 also use image signals or the digital images to form evaluation images Which have an appearance that correspond to scene images stored in 40 imaging system 10 and are adapted for presentation on dis the digital image With Which the metadata is associated. The play 30. This alloWs users of imaging system 10 to use a metadata can also comprise an instruction to incorporate a display such as display 30 to vieW images that correspond to scene images that are available in imaging system 10. Such images can include images that have been captured by scene particular message into a digital image When the digital image is presented. Such a message can be a text message that is intended to be shoWn or rendered When the digital image is presented or rendered. The metadata can also include audio input capture device and/ or that Were otherWise obtained such as by Way of communication module 54 and stored in a signals. The metadata can further include digital image data. memory such as memory 40 or removable memory 48. 45 Display 30 can comprise, for example, a color liquid crys In one embodiment of the invention, Where digital Zoom is used to form the image from a subset of the captured image, the metadata can include image data from portions of an image that are not incorporated into the subset of the digital image that is used to form the digital image. The metadata can also include any other information entered into imaging sys tem 10, sensed by imaging system 10 or determined in Whole or in part by imaging system 10. tal display (LCD), organic light emitting display (OLED) also 50 knoWn as an organic electro-luminescent display (OELD) or other type of video display. Display 30 can be external as is shoWn in FIG. 2, or it can be internal for example used in a vieW?nder system 38. Alternatively, imaging system 10 can have more than one display 30 With, for example, one being 55 external and one internal. A captured digital image and optional metadata, can be Signal processor 26 and/or controller 32 can also cooperate stored as an archival image or used for other purposes as to generate other images such as text, graphics, icons and other information for presentation on display 30. This can described herein. A digital image can be stored, for example, in a compressed form. For example, Where the digital image comprises a sequence of still images, the still images can be stored in a compressed form such as by using the JPEG (Joint alloW interactive communication betWeen controller 32 and a 60 mation to the user of imaging system 10 and the user of imaging system 10 using user input system 34 to interactively provide information to imaging system 10. Imaging system Photographic Experts Group) ISO 10918-1 (ITU-T.81) stan dard. This JPEG compressed image data is stored using the so-called “Exif” image format de?ned in the Exchangeable Image File Format version 2.2 published by the Japan Elec tronics and Information Technology Industries Association JEITA CP-3451. Similarly, other compression systems such user of imaging system 10, With display 30 providing infor 10 can also have other displays such as a segmented LCD or 65 LED display (not shoWn) Which can also permit signal pro cessor 26 and/or controller 32 to provide information to user. This capability is used for a variety of purposes such as US 8,416,317 B2 10 establishing modes of operation, entering control settings, higher degrees of focus can be located by identifying portions user preferences, and providing Warnings and instructions to in the stored image that have a greater amount of high fre quency data Which in turn is indicative of the level of focus. a user of imaging system 10. In the embodiments of FIGS. 1 and 2, imaging system 10 has an optional audio system 70 having an input transducer in the form of a microphone 72 that receives sonic energy and generates signals that are provided to audio processing cir Typically the various portions 102-118 of calibration image 100 Will depict scene elements With greater and lesser degrees of focus and thus controller 32 or signal processor 26 Will identify one area (eg area 116) as having the greatest degree of focus. It Will be appreciated that in other embodi ments, controller 32 can be adapted to simply identify the ?rst cuitry 74. Audio processing circuitry 74 is adapted to convert the signals received from microphone 72 into an electronic audio signal representing the pattern of sonic energy incident upon the transducer. Audio processing circuitry 74 is further portion that has a level of focus that is above a threshold and to select that portion. Other approaches can also be used. Range?nder 27 is used to measure a calibration focus dis tance from the digital camera 12 to selected portion 116 using adapted to receive signals from controller 32 and to cause speaker 76 to generate audible sounds. Other systems such as knoWn circuits, lights and actuators for generating visual signals, audio signals, vibrations, haptic feedback and other for example multi-spot range ?nding to measure the distance forms of signals can also be incorporated into imaging system 10 for use in providing information, feedback and Warnings to associated With the “spot” (step 86). In one embodiment, the user of imaging system 10. controller 32 and/or signal processor 26 divide calibration from digital camera 12 to a portion of the ?eld of vieW Typically, display 30 has less imaging resolution than scene image sensor 24. Accordingly, signal processor 26 reduces the resolution of a captured or stored image signal or 20 HoWever, this is not strictly necessary so long as there exists a general correspondence betWeen the siZe and location of the identi?ed portion 116 and the portion of the ?eld of vieW used digital image When forming evaluation images adapted for presentation on display 30. DoWn sampling and other con ventional techniques for reducing the overall imaging reso lution can be used. For example, resampling techniques such by range?nder 27 in measuring the distance to the designated 25 as are described in commonly assigned US. Pat. No. 5,164, 831 “Electronic Still Camera Providing Multi-Format Stor In one embodiment, calibration image 100 is captured at et al. on Mar. 15, 1990, can be used. The evaluation images 30 evaluation images can be adapted to be provided to an optional display driver 28 that can be used to drive display 30. Alternatively, the evaluation images can be converted into signals that can be transmitted by signal processor 26 in a form that directly causes display 30 to present the evaluation images. Where this is done, display driver 28 can be omitted. made for portions of the ?eld of vieW associated With each range?nding “spot” in the scene. This reduces the likelihood that the composition of the scene can change betWeen the time of image capture and the time of range?nding measurements. However, it Will be appreciated that minor time variations Will be acceptable in certain applications. 35 A focus correlation is then determined based upon the calibration distance and the focus distance setting used to capture the calibration image (step 88). The focus correlation can be any type of data, pro grammic, algorithmic, mathemati cal or logical structure that associates different range?nder distance values With different focus settings for lens system FIG. 4 shoWs a block diagram of a ?rst method for auto calibrating the range?nder based autofocus system in Which a digital camera 12 is turned ON Without an instruction to immediately capture an archival image. As is illustrated in portion to measure a distance to that portion. about the same time that range?nding measurements are age Of FullAnd Reduced Resolution Images” ?led by Kuchta can optionally be stored in a memory such as memory 40. The image 100 into portions that correspond directly to the por tions associated With each “spot” used by range?nder 27. 40 23. FIG. 6 shoWs one example of a focus correlation expressed, in this example, in the form of a tWo-dimensional FIG. 4, When controller 32 detects that a user has taken an action to activate digital camera 12 (step 80), controller 32 look-up table (LUT). In this example, taking lens system 23 causes a calibration image 100 of a ?rst ?eld of vieW to be can be set into one often focus distance settings focusing light automatically captured (step 82). FIG. 5 illustrates one example of such a calibration image 100. 45 from a different range of distances onto scene image sensor 24 associated With a range of focus distances. Each of the ten Controller 32 can determine a taking lens setting for use in focus distance settings is also associated With a range?nding capturing calibration image 100 by setting lens system 23 to distance. In a typical camera of the prior art this correlation set is ?xed for all images. HoWever, as noted above, such an a predetermined ?rst focus distance setting Which can be for example a position at a middle of an adjustable range of taking lens system 23. HoWever, in other embodiments con troller 32 can select any of the other focus distance settings. Alternatively, controller 32 can be adapted to capture an 50 image using Whatever focus distance setting lens system 23 is plurality of focus correlations available. For this example, controller 32 can select from tWo other focus correlation set at the moment that controller 32 detects some condition indicating that digital camera 12 is to be activated. Signal processor 26 and/or controller 32 portion the cali bration image 100 into portions 102-118 and identify one of the portions 102-118 of calibration images 100 as having a preferred level of focus as that term is understood in the art 55 (step 84). Such focus level can, for example, be determined by 60 distinctiveness or outline found in the image, or using any other knoWn metric for analyZing image focus. The preferred level of focus can be de?ned in relative terms by Way of frequency domain, portions of calibration image 100 having LUTs. This plurality is depicted herein as three different LUTs in FIGS. 6, 7 and 8. Controller 32 selects from among the available LUTs by looking for the focus correlation that provides a combination of focus setting and a capture focus distance that most closely correlates to the focus setting used to capture the calibration image and the calibration focus distance determined by range?nder 27 for the portion of the image identi?ed as having the predetermined degree of focus at that focus setting. For example, if taking lens system 23 Was focused at setting 3 during capture of the calibration image examining any or all of the level of contrast, clarity, detail, comparison With other portions. In one speci?c example, Where calibration image 100 is stored by compression in the arrangement does not permit digital camera 12 to adjust to variations in operation due to environmental conditions or mechanical variations. Accordingly, in this embodiment, digital camera 12 has a 65 and the determined focus calibration distance Was 0.6 meters, then controller 32 Would select the focus correlation depicted in FIG. 8.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project