Image Processing Concepts and Fundamentals

As many of my friends wanted me to write something about Image processing, when and how it is done and basic fundamentals of Image Processing, here I am with you to explain the Concepts and Fundamentals  Image Processing.

Introduction

Modern digital technology has made it possible to manipulate multi-dimensional signals with systems that range from simple digital circuits to advanced parallel computers. The goal of this manipulation can be divided into three categories:

* Image Processing image in -> image out

* Image Analysis image in -> measurements out

* Image Understanding image in -> high-level description out

We will focus on the fundamental concepts of image processing. Space does not permit us to make more than a few introductory remarks about image analysis.

Further, we will restrict ourselves to two-dimensional (2D) image processing although most of the concepts and techniques that are to be described can be extended easily to three or more dimensions. Readers interested in either greater detail than presented here or in other aspects of image processing are referred to. We begin with certain basic definitions. An image defined in the “real world” is considered to be a function of two real variables, for

example, a(x,y) with a as the amplitude (e.g. brightness) of the image at the real coordinate position (x,y). An image may be considered to contain sub-images sometimes referred to as regions-of-interest, ROIs, or simply regions. This concept reflects the fact that images frequently contain collections of objects each of which can be the basis for a region. In a sophisticated image processing system it should be possible to apply specific image processing operations to selected regions. Thus one part of an image (region) might be processed to suppress motion blur while another part might be processed to improve color rendition.

The amplitudes of a given image will almost always be either real numbers or integer numbers. The latter is usually a result of a quantization process that converts a continuous range (say, between 0 and 100%) to a discrete number of levels. In certain image-forming processes, however, the signal may involve photon counting which implies that the amplitude would be inherently quantized. In other image forming procedures, such as magnetic resonance imaging, the direct physical measurement yields a complex number in the form of a real magnitude and a real phase.

Digital Image Definitions

A digital image a[m,n] described in a 2D discrete space is derived from an analog image a(x,y) in a 2D continuous space through a sampling process that is frequently referred to as digitization. The mathematics of that sampling process will be described in Section 5. For now we will look at some basic definitions associated with the digital image. The effect of digitization is shown in Figure 1.

The 2D continuous image a(x,y) is divided into N rows and M columns. The intersection of a row and a column is termed a pixel. The value assigned to the integer coordinates [m,n] with {m=0,1,2,…,M-1} and {n=0,1,2,…,N-1} is a[m,n]. In fact, in most cases a(x,y)–which we might consider to be the physical signal that impinges on the face of a 2D sensor–is actually a function of many variables including depth (z), color (

Digital Image Definitions

A digital image a[m,n] described in a 2D discrete space is derived from an analog image a(x,y) in a 2D continuous space through a sampling process that is frequently referred to as digitization. The mathematics of that sampling process will be described in Section 5. For now we will look at some basic definitions associated with the digital image. The effect of digitization is shown in Figure 1.

The 2D continuous image a(x,y) is divided into N rows and M columns. The intersection of a row and a column is termed a pixel. The value assigned to the integer coordinates [m,n] with {m=0,1,2,…,M-1} and {n=0,1,2,…,N-1} is a[m,n]. In fact, in most cases a(x,y)–which we might consider to be the physical signal that impinges on the face of a 2D sensor–is actually a function of many variables including depth (z), color (), and time (t). Unless otherwise stated, we will consider the case of 2D, monochromatic, static images in   this paper.

), and time (t). Unless otherwise stated, we will consider the case of 2D, monochromatic, static images in this paper.

Figure 1: Digitization of a continuous image. The pixel at coordinates [m=10, n=3] has the integer brightness value 110

The image shown in Figure 1 has been divided into N = 16 rows and M = 16 columns. The value assigned to every pixel is the average brightness in the pixel rounded to the nearest integer value. The process of representing the amplitude of the 2D signal at a given coordinate as an integer value with L different gray levels is usually referred to as amplitude quantization

Common Values

There are standard values for the various parameters encountered in digital image processing. These values can be caused by video standards, by algorithmic requirements, or by the desire to keep digital circuitry simple. Table 1 gives some commonly encountered values

Parameter	Symbol	Typical values
Rows	N	256,512,525,625,1024,1035
Columns	M	256,512,768,1024,1320
Gray Levels	L	2,64,256,1024,4096,16384

Quite frequently we see cases of M=N=2^K where {K = 8,9,10}. This can be motivated by digital circuitry or by the use of certain algorithms such as the (fast) Fourier transform.

The number of distinct gray levels is usually a power of 2, that is, L=2^B where B is the number of bits in the binary representation of the brightness levels. When B>1 we speak of a gray-level image; when B=1 we speak of a binary image. In a binary image there are just two gray levels which can be referred to, for example, as “black” and “white” or “0″ and “1″.

Characteristics of Image Operations

• Types of operations
• Types of neighborhoods

There is a variety of ways to classify and characterize image operations. The reason for doing so is to understand what type of results we might expect to achieve with a given type of operation or what might be the computational burden associated with a given operation.

Types of operations

The types of operations that can be applied to digital images to transform an input image a[m,n] into an output image b[m,n] (or another representation) can be classified into three categories as shown in Table 2.

Operation	Characterization	Generic Complexity/Pixel
* Point	- the output value at a specific coordinate is dependent only on the input value at that same coordinate.	constant
* Local	- the output value at a specific coordinate is dependent on the input values in the neighborhood of that same coordinate.	P2
* Global	- the output value at a specific coordinate is dependent on all the values in the input image.	N2

Types of neighborhoods

Neighborhood operations play a key role in modern digital image processing. It is therefore important to understand how images can be sampled and how that relates to the various neighborhoods that can be used to process an image.

* Rectangular sampling – In most cases, images are sampled by laying a rectangular grid over an image as illustrated in Figure 1. This results in the type of sampling shown in Figure 3ab.

* hexagonal sampling – An alternative sampling scheme is shown in Figure 3c and is termed hexagonal sampling.

Both sampling schemes have been studied extensively and both represent a possible periodic tiling of the continuous image space. We will restrict our attention, however, to only rectangular sampling as it remains, due to hardware and software considerations, the method of choice.

Local operations produce an output pixel value b[m=m_o,n=n_o] based upon the pixel values in the neighborhood of a[m=m_o,n=n_o]. Some of the most common neighborhoods are the 4-connected neighborhood and the 8-connected neighborhood in the case of rectangular sampling and the 6-connected neighborhood in the case of hexagonal sampling illustrated in Figure 3.

Tools

Certain tools are central to the processing of digital images. These include mathematical tools such as convolution, Fourier analysis, and statistical descriptions, and manipulative tools such as chain codes and run codes.

1. Convolution

Properties of Convolution

There are a number of important mathematical properties associated with convolution.

* Convolution is commutative.

* Convolution is associative.

* Convolution is distributive.

2. Fourier Transforms

The Fourier transform produces another representation of a signal, specifically a representation as a weighted sum of complex exponentials.

3.Contour Representations

• Chain code
• Chain code properties
• Crack code
• Run codes

When dealing with a region or object, several compact representations are available that can facilitate manipulation of and measurements on the object. In each case we assume that we begin with an image representation of the object as shown in Figure 8a,b. Several techniques exist to represent the region or object by describing its contour.

Chain code

This representation is based upon the work of Freeman . We follow the contour in a clockwise manner and keep track of the directions as we go from one contour pixel to the next. For the standard implementation of the chain code we consider a contour pixel to be an object pixel that has a background (non-object) pixel as one or more of its 4-connected neighbors.

Chain code properties

* Even codes {0,2,4,6} correspond to horizontal and vertical directions; odd codes {1,3,5,7} correspond to the diagonal directions.

* Each code can be considered as the angular direction, in multiples of 45deg., that we must move to go from one contour pixel to the next.

* The absolute coordinates [m,n] of the first contour pixel (e.g. top, leftmost) together with the chain code of the contour represent a complete description of the discrete region contour.

* When there is a change between two consecutive chain codes, then the contour has changed direction. This point is defined as a corner.

Crack code

An alternative to the chain code for contour encoding is to use neither the contour pixels associated with the object nor the contour pixels associated with background but rather the line, the “crack”, in between.

Run codes

A third representation is based on coding the consecutive pixels along a row–a run–that belong to an object by giving the starting position of the run and the ending position of the run.

4.Perception

Many image processing applications are intended to produce images that are to be viewed by human observers (as opposed to, say, automated industrial inspection.) It is therefore important to understand the characteristics and limitations of the human visual system–to understand the “receiver” of the 2D signals. At the outset it is important to realize that

1) the human visual system is not well understood,

2) no objective measure exists for judging the quality of an image that corresponds to human assessment of image quality,

3) the “typical” human observer does not exist. Nevertheless, research in perceptual psychology has provided some important insights into the visual system.

• Brightness Sensitivity
• Spatial Frequency Sensitivity
• Color Sensitivity
• Optical Illusions

Image Sampling

Converting from a continuous image a(x,y) to its digital representation b[m,n] requires the process of sampling.

1.Histogram-based Operations

• Contrast stretching
• Equalization
• Other histogram-based operations

An important class of point operations is based upon the manipulation of an image histogram or a region histogram. The most important examples are described below.

Contrast stretching

Frequently, an image is scanned in such a way that the resulting brightness values do not make full use of the available dynamic range.

Equalization

When one wishes to compare two or more images on a specific basis, such as texture, it is common to first normalize their histograms to a “standard” histogram. This can be especially useful when the images have been acquired under different circumstances. The most common histogram normalization technique is histogram equalization where one attempts to change the histogram through the use of a function.

Mathematics-based Operations

• Binary operations
• Arithmetic-based operations

Binary operations

Operations based on binary (Boolean) arithmetic form the basis for a powerful set of tools

The implication is that each operation is applied on a pixel-by-pixel basis.

Arithmetic-based operations

The gray-value point operations that form the basis for image processing are based on ordinary mathematics like

ADD, SUB, MUL, DIV, LOG, EXP,  SQRT, TRIG., INVERT etc

Convolution-based Operations

• Background
• Convolution in the spatial domain
• Convolution in the frequency domain

Convolution, the mathematical, local operation central to modern image processing. The basic idea is that a window of some finite size and shape–the support–is scanned across the image. The output pixel value is the weighted sum of the input pixels within the window where the weights are the values of the filter assigned to every pixel of the window itself. The window with its weights is called the convolution kernel.

Background

In a variety of image-forming systems an appropriate model for the transformation of the physical signal a(x,y) into an electronic signal c(x,y) is the convolution of the input signal with the impulse response of the sensor system. This system might consist of both an optical as well as an electrical sub-system. If each of these systems can be treated as a linear, shift-invariant (LSI) system then the convolution model is appropriate.

Techniques to solve specific image processing problems.

Shading Correction

• Model of shading
• Estimate of shading

The method by which images are produced–the interaction between objects in real space, the illumination, and the camera–frequently leads to situations where the image exhibits significant shading across the field-of-view. In some cases the image might be bright in the center and decrease in brightness as one goes to the edge of the field-of-view. In other cases the image might be darker on the left side and lighter on the right side. The shading might be caused by non-uniform illumination, non-uniform camera sensitivity, or even dirt and dust on glass (lens) surfaces. In general this shading effect is undesirable. Eliminating it is frequently necessary for subsequent processing and especially when image analysis or image understanding is the final goal.

Basic Enhancement and Restoration Techniques

• Unsharp masking
• Noise suppression
• Distortion suppression

The process of image acquisition frequently leads (inadvertently) to image degradation. Due to mechanical problems, out-of-focus blur, motion, inappropriate illumination, and noise the quality of the digitized image can be inferior to the original. The goal of enhancement is– starting from a recorded image c[m,n]–to produce the most visually pleasing image â[m,n]. The goal of restoration is–starting from a recorded image c[m,n]–to produce the best possible estimate â[m,n] of the original image a[m,n]. The goal of enhancement is beauty; the goal of restoration is truth.

The measure of success in restoration is usually an error measure between the original a[m,n] and the estimate â[m,n]: E{â[m,n], a[m,n]}. No mathematical error function is known that corresponds to human perceptual assessment of error. The mean-square error function is commonly used because:

1. It is easy to compute;

2. It is differentiable implying that a minimum can be sought;

3. It corresponds to “signal energy” in the total error, and;

4. It has nice properties vis à vis Parseval’s theorem,

Unsharp masking

A well-known technique from photography to improve the visual quality of an image is to enhance the edges of the image. The technique is called unsharp masking. Edge enhancement means first isolating the edges in an image, amplifying them, and then adding them back into the image. Examination of following picture shows that the Laplacian is a mechanism for isolating the gray level edges.

Noise suppression

The techniques available to suppress noise can be divided into those techniques that are based on temporal information and those that are based on spatial information. By temporal information we mean that a sequence of images {a_p[m,n] | p=1,2,…,P} are available that contain exactly the same objects and that differ only in the sense of independent noise realizations.

Distortion suppression

The model presented above–an image distorted solely by noise–is not, in general, sophisticated enough to describe the true nature of distortion in a digital image. A more realistic model includes not only the noise but also a model for the distortion induced by lenses, finite apertures, possible motion of the camera and/or an object, and so forth. One frequently used model is of an image a[m,n] distorted by a linear, shift-invariant system h_o[m,n] (such as a lens) and then contaminated by noise k[m,n].

The restoration procedure that is based on linear filtering coupled to a minimum mean-square error criterion again produces a Wiener filter :

Segmentation

• Thresholding
• Edge finding
• Binary mathematical morphology
• Gray-value mathematical morphology

In the analysis of the objects in images it is essential that we can distinguish between the objects of interest and “the rest.” This latter group is also referred to as the background. The techniques that are used to find the objects of interest are usually referred to as segmentation techniques – segmenting the foreground from background. In this section we will two of the most common techniques–thresholding and edge finding– and we will present techniques for improving the quality of the segmentation result. It is important to understand that:

* there is no universally applicable segmentation technique that will work for all images, and,

* no segmentation technique is perfect.

Thresholding

This technique is based upon a simple concept. A parameter theta called the brightness threshold is chosen and applied to the image a[m,n].

Edge finding

Thresholding produces a segmentation that yields all the pixels that, in principle, belong to the object or objects of interest in an image. An alternative to this is to find those pixels that belong to the borders of the objects. Techniques that are directed to this goal are termed edge finding techniques.

Hope you find it useful.