Computer Vision Week 2 Image Formation
Overview of image formation
- Light from a radiation source is reflected by surfaces in the world..
- Refleted light passes through some optics and is recorded by a sensor
- Images are formed when a sensor registers radiation that has interacted with physical objects
Physics of image formation
- ligth
- reflectance
- optics
Ingredients of image formation
The image that is formed is affected by two sets of parameters
Radiometric parameters
Determine the intensity/colour of a given location in the image
- illumination(type, number, location, intensity, colour-spectrum)
- surface reflectance properties(material, orientation)
- sensor properties(sensitivity to different electromagnetic frequencies)
Geometric parameters
Determine where on the image a scene point appears
- camera position and orientation in space
- camera optics(e.g. focal length)
- projection geometry(mapping from 3D to 2D)
Light and Colour
visible spectrum
At the earth’s surface the intensity of the electromagnetic radiation emanating from the sun has a peak within the 400-700nm range.
- The human eye has evolved a specific sensitivity to this part of the electromagnetic spectrum.
- Hence, visible light is that part of the electromagnetic spectrum with a wavelength (λ) between 400 and 700nm.
- Cameras and Computer Vision systems also concentrate on this part of the spectrum (but not exclusively, e.g. infra-red cameras for detecting body heat, X-rays for imaging inside the body).
Colour Perception
- The radiation that drives the human construct of colour, is fundamentally colour less.
- The sensation of colour is determined by the human visual system, based on the product of light and reflectance.
- However, it is a convenient short-hand to refer to electormagnetic radiation as having colour, e.g. to say “red-light” or “blue-light”.
Luminance
Light is differentially reflected at each wavelength, which gives objects their natural colours.
albedo = fraction of light reflected at a particular wavelength
An object looks green because is absorbes red and blue light leaving
more green in the reflected light.
Illuminance
Light is produced in different amounts at different wavelengths by each light source.
Colour mixing
Mixing light: additive
e.g. a green light plus a blue light plus a red light gives light containing a broad spectrum of light, i.e. white.
The illumination from different light sources adds.
Mixing pigments: subtractive
e.g. a green pigment plus a blue pigment plus a red pigment gives a pigment that absorbs light over a broad spectrum, leaving black.
The reflection from different surfaces subtracts.
Measuring surface properties
The biological vision system percetives the colour of surfaces in the world, not the coulour of the light entering the eyes.
This is an ill-posed problem: we record L but need to recover R and don’t know E
$$ L(x,y,\lambda)=fn(E(x,y,\lambda),R(x,y,\lambda)) $$
- Luminance(L) amount of light striking the sensor, depends on
- Illuminance(E) amount of light striking the surface as well as
- Reflectance(R) which depends on material properties
Color Constancy
Artificial
To recover the surface colour of a particular location,$R(x,y,\lambda))$,we need to know the colour of the illuminationn at that point, $E(x,y,\lambda))$
many ways of approximating E have been suggested:
- Average reflectance across scene is known(often fails)
- Fixing brightest image patch to be white
- Gamut(collection of all colours) falls within known arrange
- Known reference colour (colour chart, skin colour)
- Specular reflections have the colour of the illumination
None of these are work particularly well
Biological
However, the human visual system does seem able to recover surface colour, since despite large changes in illumination (and consequently the intensity spectrum that enters our eyes), we usually experience the colour of an object as being constant.
We are not normally aware of this variation because colour constancy mechanisms discount the effects of illumination, and infer the colour of the objects.
Focusing Light
Light spreads out from a point.
Without some kind of optics each location on the sensor will register light coming from many different points in the world.
No image will be formed.
Optics
Pinhole Camera
Restricts the flow of light (using a small hole) so that only one ray from each point in the world reaches the sensor.
“Focus” means that all rays coming from a scene point converge into a single image point.
“Exposure” is the time needed to allow enough light through to form an image (the smaller the aperture, the longer the exposure time). The longer the exposure the more blurred an image is likely to be.
- mall pinhole: sharp focus but dim image (long exposure time)
- Large pinhole: brighter image (shorter exposure) but blurred
To produce an image that is both bright and in focus requires a lens
Lensed Camera
With a large pinhole, light rays from the same point project to different locations on the image. The image is blurred.
By focusing rays from the same point onto a single image location, a lens can keep the image sharp while gathering more light.
Cost: image focused for only a restricted range of object positions