Intelligent Photography

in a digital age February 11, 2008

Goals
Learn a little about photography Understand how basic camera systems work

AF auto focus AE auto exposure

Recognize intelligent features Show how neural nets and fuzzy logic are used in digital cameras

Outline
Brief history of the camera How a digital camera works Challenges in digital photography Intelligent features in todays digital cameras Case studies

Using fuzzy logic and neural nets

History of the Camera

Pinhole cameras/camera obscura

1000 AD Ibn al-Haytham

History of the Camera

Wet Plate Cameras

1100 early 1800s

Discovery of silver nitrate, silver chloride, diaphragm, photochemical effect

1825-1827 Joseph Nicephore Niepce

The first photographs Used a pewter place coated with bitumen and exposed to light

History of the Camera

Wet plate cameras

1836 Louis Jacques Daguerre

Daguerreotype camera Copper plate coated with silver, treated with iodine vapor Developed using mercury vapor and fixed with a salt solution

1840 William Fox Talbot

Calotype camera

History of the Camera

Dry plate cameras

1855 von Monckhoven

Collodian dry plates

1871 Richard Maddox

Gelatin dry plates

Began to rival wet plates Pros: photographers could use commercially made dry plates instead of preparing their own emulsions in a mobile dark room Allowed cameras to be small enough to be hand-held Shortened exposure times lead to invention of the shutter

History of the Camera

Photographic Film

1885 George Eastman

invented paper film

1888 Kodak camera

Simple box camera Fixed-focus lens with single shutter speed Low cost brought photography to the masses Preloaded with 100 exposures, had to be returned to the factory for processing and reloading

1889 celluloid film 1910 Kodak Brownie

Simple and inexpensive enough to introduce the snapshot On sale until the 1960s

History of the Camera

First Modern Cameras

1925 Leica 1
Introduced 35mm film

1928 Rolleiflex medium format TLR 1933 Exakta SLR

History of the Camera

The Modern Camera

1938 Super Kodak Six-20

First auto-exposure camera

1948 Polaroid Model 95

First instant picture camera

1960 TTL metering 1977 Konica C35AF

First mass produced AF point and shoot

1978 Polaroid SX-70

First AF SLR

1981 Pentax ME-F

First 35mm AF SLR Utilized a motorized lens

1986 Minolta Maxxum 7000

Integrated AF system

History of the Camera

The Digital Camera

1986 Canon RC-701

First analog digital camera First used by USAToday for the 1986 World Series

Poor quality, could transmit pictures over the phone

1988 Canon RC-250 Xapshot

Analog camera marketed to consumers

1990s introduction of the digital P&S 1999 Nikon D1

2.74MP, first professional digital SLR

2003 Canon 300D (digital rebel)

First DSLR under $1000

2007 Canon 40D

First live view DSLR

How a digital camera works

Digital/Film cameras use the same principles

Lens Aperture Shutter Film/Sensor

How a digital camera works

Lenses

Fixed-focus, fixed-zoom (disposable cameras) Optical zoom Removable lenses

Primes Zooms

Image stabilized lenses

How a digital camera works

Aperture the hole or opening in a lens through which light is admitted

Controls depth of field Measured in f-stops

How a digital camera works

Shutter

Opens to expose the film or sensor to light Closes to stop exposure Shutter times measured in seconds
i.e. 1/100, 1/50, 1/20, 1/10, 1/5, , 1, 2, 4, 8 Also measured in stops of lights

How a digital camera works

Film

35mm Windable ISO speed films sensitivity to light

ISO 400, 200, 100 Higher ISO, faster the film. Lower ISO, slower film. Higher ISO, film is more grainy

Sensors

Like film, different sizes

Crop sensors/full frame sensors ISO sensitivity like film Two types

CCD charge coupled device, analog shift register CMOS - complementary metaloxidesemiconductor digital recorder Requires less power, less susceptible to noise

How a digital camera works

Exposure

combination of shutter speed, aperture, and film speed (ISO) a combination of shutter speed, aperture, and ISO that provides a correctly exposed image Underexposure too dark Overexposure washed out

Correct Exposure

Example: filling a glass of water from a faucet

Exposure
For any given situation, there are many creatively correct exposures Seven different kinds of creative exposure

Story telling small apertures (f/32, f/22, f/16) Isolation large apertures (f/2.8, f/4, f/5.6) Who cares in between apertures, depth of field isnt a concern Macro out of focus backgrounds (bokeh) Freeze action fast shutter speeds (1/1000, 1/500, 1/250) Panning medium shutter speeds (1/60, 1/30) Imply motion snow shutter speeds (1/4, , 1)

Exposure
A Stop of light a doubling or halving of light

One stop of light is equal to another

One f-stop One ISO stop One shutter stop

Can trade one stop for another to keep the same exposure Sunny f/16 rule: on a sunny day, f/16 and 1/ISO to give correct exposure

Challenges in Digital Photography

Technological issues

File types Auto Focus Auto Exposure Noise Reduction Sharpness Contrast White Balance

Challenges in Digital Photography

File Types:

JPEG joint photographic experts group

Most commonly used format for web and digital cameras

Pros used everywhere, small file sizes, good compression, fit more on a memory card, faster download times Cons lossy compression leads to data loss and image degradation from resaving Pros lossless image compression, no image degradation Cons not universally supported, large image sizes, still lose some information and ability to process photos

TIFF tagged image file format

RAW camera proprietary format

Each camera has its own format

Pros full control over image processing, smaller files than TIFF, completely lossless, no compression Cons almost no support, formats constantly changing, must do all processing manually

Challenges in Digital Photography

Auto-Focus

Active methods independent of the optical system

IR and ultrasounds Work in low light, low contrast situations, will not work through glass or water

Passive methods uses analysis of the image itself

Phase Detection and Contrast Measurement More accurate, will work through glass, bad in low light or low contrast situations, allows for high numbers of focus points Can only be used with digital cameras

Challenges in Digital Photography

Auto Focus how does it work?

Active
1. 2. 3. 4. Sends a signal towards target Measures returning signal Calculates distance Drives lens to desired position

Challenges in Digital Photography

Auto Focus how does it work?

Passive
1. 2. 3. Looks at current image Move lens slightly, compare the two images (sharpness, contrast) If better, keep moving. If worse, go other direction

Challenges in Digital Photography

Auto Exposure choosing the correct exposure

Metering Techniques
Spot only measures in a 2-3 degree cone in the center of image Center Weighted weights the center 50-60% of the image Average averages the entire scene Partial only uses a certain region (usually the center) Evaluative uses information from several zones and combines the info to find the best exposure Scene Modes gives the camera an idea of what it is seeing so it can apply special rules when metering (night, indoors, museum, snow, sports)

Challenges in Digital Photography

Auto Exposure how does it work?
1. Exposure mode measures incoming light 2. Calculates exposure so that scene is approximately 18% gray 3. Calculates minimum shutter speed based on lens focal length 4. Calculates required aperture for min shutter speed 5. If not possible, add stops from ISO 6. If necessary, shoot with aperture wide open

Challenges in Digital Photography

1. 2. 3. 4. 5. 6. 18% gray card, AE White card, AE White card, ME Black card, ME Black Card, ME Black, Gray, and White cards, AE

Intelligent Cameras
Todays digital cameras have many intelligent capabilities

Fuzzy Logic systems

Auto exposure decision making Focus point selection Passive auto focus

Neural systems
Evaluative or matrix metering Face detection

Intelligent Cameras
Auto Exposure Decision Making

Once an exposure has been calculated (using any of the metering methods) must then choose the creatively correct exposure Fuzzy rules based on
Available ISOs, shutter speeds, and apertures Weights or priorities given to different settings

Example: Auto depth of field, action, museum modes

Intelligent Cameras
Focus Point Selection

Most newer cameras have multiple focus points Must choose which focus point you intended the camera to use Fuzzy logic rules:
If 1 is near and 2 and 3 is far, 1 is somewhat likely If 1 is near and 2 is near and 3 is far, 1 and 2 are highly likely If 1 and 3 are far and 2 is near, 2 is highly likely Ect.

Intelligent Cameras: Case Study

Passive Auto-focus using Fuzzy Logic

Image is compared as lens focuses to choose the best image Using a sharpness measure
Define over a portion of the image Use fuzzy membership functions and rules to drive the lens

Intelligent Cameras: Case Study

Passive Auto-focus

Fuzzy rules
If sharpness is increasing fast, move lens fast in same direction If sharpness is increasing slowly, move lens slowly in same direction If sharpness is not changing, do not move lens If sharpness is decreasing, move lens lowly in opposite direction

Use Matlabs Fuzzy Logic toolbox to easily model this system

Case Study
Face Detection via neural network