3D imaging

The 3D Image -- Depth Maps

The simplest and most convenient way of representing and storing the depth measurements taken from a scene is a depth map.

A depth map is a two-dimensional array where the x and y distance information corresponds to the rows and columns of the array as in an ordinary image, and the corresponding depth readings (z values) are stored in the array's elements (pixels).

Depth map is like a grey scale image except the z information (float - 32 bytes) replaces the intensity information.

 

Fig. 32 Artificial depth maps

 

Fig. 33 Real depth maps

Why use 3D data?

An 3D image containing has many advantages over its 2D counterpart:

Explicit Geometry

--

• 2D images give only limited information the physical shape and size of an object in a scene.
• 3d images express the geometry in terms of three-dimensional coordinates.

e.g Size (and shape) of an object in a scene can be straightforwardly computed from its three-dimensional coordinates.

Recent technological advances ( e.g. in camera optics, CCD cameras and laser rangefinders) have made the production of reliable and accurate three-dimensional depth data possible.

Consequently many three-dimensional data acquisition systems have been developed.

Methods of Acquisition

Laser Ranging Systems

Laser ranging works on the principle that the surface of the object reflects laser light back towards a receiver which then measures the time (or phase difference) between transmission and reception in order to calculate the depth.

Most laser rangefinders:

• Work at long distances (greater than )
• Consequently their depth resolution is inadequate for detailed vision tasks.
• Shorter range systems exist but still have an inadequate depth resolution (1cm at best) for most practical industrial vision purposes.

Structured Light Methods

Basic idea:

• Project patterns of light (grids, stripes, elliptical patterns etc.) onto an object.
• Surface shapes are then deduced from the distortions of the patterns that are produced on Object's Surface.
• Knowing relevant camera and projector geometry, depth can be inferred by triangulation.

• Many methods have been developed using this approach.
• Major advantage -- simple to use.
• Low spatial resolution -- patterns become sparser with distance.
• Some close range (4cm) sensors exist with good depth resolution (around 0.05mm) but have very narrow field of view and close range of operation.

Moire Fringe Methods

The essence of the method is that a grating is projected onto an object and an image is formed in the plane of some reference grating as shown in Fig. 34.

The image then interferes with the reference grating to form Moire fringe contour patterns which appear as dark and light stripes, as demonstrated by Fig. 35. Analysis of the patterns then gives accurate descriptions of changes in depth and hence shape.

 

Fig.  A moire projection system

 

Fig. 35 Moire fringe patterns

NOTE: Ambiguities arise in interrogating the fringe patterns.

• It is not possible to determine whether adjacent contours are higher or lower in depth.
• Resolve by moving one of the gratings and taking multiple Moire images.
• Reference grating can also be omitted and its effect can be simulated in software.

Moire fringe methods are capable of producing very accurate depth data (resolution to within about 10 microns) but the methods have certain drawbacks.

• Methods are relatively computationally expensive.
• Surfaces at a large angle are sometimes unmeasurable -- fringe density becomes too dense.

Shape from Shading Methods

Methods based on shape from shading employ photometric stereo techniques to produce depth measurements.

Using a single camera, two or more images are taken of an object in a fixed position but under different lighting conditions.

By studying the changes in brightness over a surface and employing constraints in the orientation of surfaces, certain depth information may be calculated.

Methods based on these techniques are not suited for general three-dimensional depth data acquisition:

• Methods are sensitively dependent on the illumination and surface reflectance properties of objects present in the scene.
• Methods only work well on objects with uniform surface texture.
• It is difficult to infer absolute depth, and only surface orientation is easily inferred.
• Methods are mostly used when it is desired to extract surface shape information.

Passive Stereoscopic Methods

Stereoscopy as a technique for measuring range by triangulation to selected locations in a scene imaged by two cameras already -- further details on general stereo configurations in Books.

The primary computational problem of stereoscopy is to find the correspondence of various points in the two images.

This requires:

• Reliable extraction of certain features (such as edges or points) from both images
• Matching of corresponding features between images.
• Both of these tasks are non-trivial and computationally complex.
• Passive stereo may not produce depth maps within a reasonable time.
• the depth data produced is typically sparse since high level features, such as edges, are used rather than points.

NOTE:

• Problems in finding and accurately locating features in each image can be hard.
• Care needed not to introduce errors.
• Depth measurements accurate to a few millimeters.
• One such passive stereo vision system is TINA developed at Sheffield University.

Active Stereoscopic Methods

The problems of passive stereoscopic techniques may be overcome by

• Illuminating the scene with a strong source of light (in the form of a point or line of light) which can be observed by both cameras.
• Known corresponding points provided in each image.
• Depth maps can then be produced by sweeping the light source across the whole scene.
• Laser light source typically employed.
• Active stereo can only be applied in controlled environments -- industrial applications.