ENGG1003 Project Description
May 17, 2019
1 Introduction
Digital images are finding increasing application to engineering problems thanks to the increased availability
of low cost digital cameras. Similarly, thermal imaging cameras are also becoming cheaper and therefore are
finding an increased usage in engineering applications.
However, digital images have limited value without further processing hence the need to use computers
to extract relevant features. This processing is achieved through software.
You will be required to develop suitable image processing algorithms and implement them in a MATLAB
program that will be able to extract information from images that will help to solve a specific engineering
problem.
2 Project Description
2.1 Medical Engineering
Research into the application of local hyperthermia directed to kill cancerous tumors is being conducted.
Delivery of sufficient thermal energy to kill tumors but leave other tissue unaffected means that feedback
of the thermal profile of a target area needs to be analysed. You are given images from a thermal camera
which captures the thermal distribution of a patient’s treatment area. Your task is to write a program in
MATLAB designed to process the images and extract features which indicate that the thermal treatment is
only affecting the tumor.
2.2 Chemical Engineering
You are assigned to a team which is developing a new process incorporating an experimental tank reactor.
The tank aims to generate homogeneous reaction rates throughout the tank in order to maximise its yield. To
this end thermal images are captured of the side of the tank. Your task is to write a program in MATLAB
designed to process the images and extract features which will identify area of accelerated reaction rates
or non-homogeneous reaction rates (as the reaction are exothermic) so that the design of the tank can be
changed to correct these problems.
2.3 Mechanical Engineering
You are given the thermal images of the backup roll on a four high rolling mill designed to produce sheet
steel for cars. Temperature gradients across the roll cause imperfections in resulting steel sheet. Your task
is to write a program in MATLAB designed to process the images and extract features which will identify
where these temperature gradients are and therefore allow the use of sprays to equalise the temperature
across the rolls thereby improving the quality of the resulting sheet steel.
2.4 Electrical Engineering
A study into transformer performance shows that a failing power transformer produces an increasing number
of hot spots. You are tasked to a team that is developing a field device which can quickly determine if a
1
transformer is likely to fail. Your task is to write a program in MATLAB designed to process the images
and extract features which will identify whether the transformer is failing and therefore should be replaced.
2.5 Mechatronics Engineering
Thermal imaging is used in search and rescue robots throughout the world. A new project aims to combine a
quadcopter with a thermal imaging camera to detect possible survivors amongst rubble. Your task is to write
a program in MATLAB designed to process the images and extract features which will identify potential
survivors so that rescuers can focus their efforts to the most promising locations.
2.6 Aerospace Engineering
Thermal imaging is used to identify the quality of a thermal panel for a launch vehicle’s heat protection.
Your task is to write a program in MATLAB designed to process the thermal images and extract features
which will identify faulty panels.
2.7 Environmental Engineering
Satellite thermal images of clouds are used to identify locations of potentially building storm activities. Your
task is to write a program in MATLAB designed to process the infrared images and extract features which
will help in identifying potentially destructive storms.
2.8 Information Technology
Many international airports employ thermal imaging cameras to detect passengers with fevers in an effort
to reduce the spread of disease. Since major disease outbreaks are rare, the airport you work for has moved
a thermal imaging camera to monitor passenger queues at a known bottleneck in the customs process. You
have been been assigned the task of analysing the images to estimate the number of passengers queued in a
given image.
2
3 Thermal Imagining Devices
There are two devices that capture the thermal images. A low resolution and high resolution camera. The
specifications of each device is as follows:
- Low resolution sensor
Sensor has a width of 80 pixels (4.6 mm) x height of 60 pixels (3.7 mm). Diagonal Field of view
63.5
o
. Detectable temperature range ?40oC to 382oC with a resolution of approximately 0.026oC.
The sensor returns an array of thermal intensities as an integer level represented by 14 bits.
Image data format is an array 80 by 60 which has each element containing integer numbers representing
the relative temperature level for each sensor pixel. - High resolution sensor
Sensor has a width of 640 pixels (4.6 mm) x height of 480 pixels (3.7 mm). Diagonal Field of view
63.5
o
. Detectable temperature range ?40oC to 382oC with a resolution of approximately 0.026oC.
The sensor returns an array of thermal intensities as an integer level represented by 14 bits.
Image data format is a run-length encoded (RLE) array containing encoded integer numbers representing
the relative temperature level for each sensor pixel.
3 - Requirements Specification
4.1 Task 1 (9 Marks): Process a low resolution sensor image with an idealised
feature
You are given an idealised test image with a single circular feature. You are to develop an algorithm and
implement that algorithm in the form of a MATLAB program that is capable of performing the following
tasks. - Find the centre of the circular feature in pixels.
- Calculate the radius of the feature in pixels.
- Calculate the area of the feature in pixels.
- Given the distance from the camera to the surface and sensor information, calculate the actual location
of the centre relative to the edge of the image, the radius of the feature and the area of the feature.
Task 1: Low Resolution Sensor Image - 20 30 40 50 60 70 80
Figure 1: Task 1: Example Image
Hint: A simplistic approach to achieve the above goals is to take advantage of the fact that the test image
only contains one single circular feature with a clearly defined edge. Hence, you could scan each line of the
image in a vertical direction until you find the first edge of the circle and then find the last vertical edge.
Combining this with a scan of the horizontal lines in the image would result in finding the extreme points
of the circular feature from which you can answer the questions that have been asked.
4
4.2 Task 2 (3.5 Marks): Process a low resolution sensor image with realistic
features
The first image was idealised for simplicity. The next image is more realistic and as such lens effects and other
ambient disturbances including thermal gradient are present in the image. As such a more sophisticated
algorithm of edge detection will be required. You are to develop an algorithm that uses Sobel edge detection
(see Appendix C) and implement it in the form of a MATLAB program that is capable of performing the
following tasks:
- Find the centre of the circular feature in pixels.
- Calculate the radius of the feature in pixels.
- Calculate the area of the feature in pixels.
- Given the distance from the camera to the surface and sensor information, calculate the actual location
of the centre relative to the edge of the image, the radius of the feature and the area of the feature.
You may use MATLAB’s built-in implementation of the Sobel algorithm (documentation: here) or implement
it yourself. Manual implementation will be worth more marks. Final marking guide TBA.
Task 2: Low Resolution Sensor Image - 20 30 40 50 60 70 80
Figure 2: Task 2: Example Image
5
4.3 Task 3: Process high resolution sensor image with multiple features
(2019) This task was removed to reduce student load in weeks 12 and 13.
6 - Appendices
A Approximation of Image Size
It is important to determine the real size of objects in an image. A simple approximation for achieving this
is to use the information provided by the camera’s manufacturer.
A.1 Field of View
A cameras field of view is nothing more than the visible cone that a camera is capable of seeing. This is
usually described as an angle in degrees.
Diagonal FOV
Sensor Viewable Area
Total Viewable Area
Distance to Object
Camera
Figure 3: Camera field of view schematic.
A.2 Object Size Estimation
With a small amount of trigonometry we see that the diagonal field of view can be combined with the
distance from the object to get the relationship between a pixel and the real object size that it represents.
dtva = 2do tan
(1)
where dtva is the diameter of the total viewable area, do is the distance from the focal point to the object
and F OV is the field of view angle in radians. Using dtva which is equivalent to the diagonal distance that
the sensor can see we can calculate the pixel to object size as follows.
(3)
where ws is the width of the sensor in mm, hs is the height of the sensor in mm, hsva is the height of the
sensor viewable area in mm and wsva is the width of the sensor viewable area in mm.
Then height of the object ho in mm and the width of the object wo are given by
where hsp is the height of the sensor in pixels, hop is the height of the object in pixels, wsp is the width of
the sensor in pixels and wop is the width of the object in pixels.
7
B Kernel Convolution
NB: This section describes a process which is almost identical to Question 4 from the Week 10 Lab.
Most image processing consists of some form of filtering. Filtering is performed via the convolution of a
matrix called a kernel with an image and producing a new image with the result of the convolution. The
kernel describes how each pixel in an image should be modified based on a set of weighted values applied to
the surrounding pixels in an image.
B.1 Illustrative Example
To illustrate the kernel convolution procedure we will select the following kernel k.
Secondly we will use the following image for our example. - 2 3 4 5 6
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
We note that the numbers represent values of luminosity in the range 0 to 255 where 0 corresponds to black
and 255 white. So essentially our image appears to contain a vertical ridge.
Now in order to apply the convolution or filter we take the kernel k and flip it both horizontally and
vertically kc.
Then we place the central element of kc over each pixel in the image. For example, placing kc’s central value
over the first pixel in the image we see that top and left elements are outside the bounds of the image and
therefore these kernel values will not be used in the next part of the calculation. - 2 3 4 5 6
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
Once we have overlaid the kernel on the image we calculate the filtered version of the pixel by summing the
product of each image pixel value with its flipped kernel value and dividing by the sum of kc’s elements that
have overlaid image pixels. For this example we have
r(1, 1) = 5 × 10 + 4 × 10 + - × 10 + 1 × 10 (8)
= 120/(5 + 4 + 2 + 1) (9)
= 10 (10)
Similarly if we are calculating the filtered value for pixel (3, 3) we have
8 - 2 3 4 5 6
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
r(3, 3) = - × 10 + 8 × 200 + 7 × 200 +
- × 10 + 5 × 200 + 4 × 200 +
- × 10 + 2 × 200 + 1 × 200
(11)
= 5580/(9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1) (12)
= 124 (13)
Hence the entire filtered matrix for this example would be - 2 3 4 5 6
- 10 55 119 155 91 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 64 127 146 83 10
C Sobel Edge Detector
An edge is described as a line at which there is a“sharp”change in pixel intensity. The definition of sharp
is user defined and very much dependent on what features and hence edge you are trying to extract from an
image. The detection of the edges therefore comes down to trying to identify a significant change in gradient
intensity throughout an image.
One of the most basic approximations to finding these gradients is to use a technique called Sobel edge
detection. The implementation of the Sobel edge detector simply performs two kernel convolutions where
the kernels applied are given as follows.
Once the two new images (Ix and Iy) resulting from the convolutions are obtained, they are combined to
give the resulting image which represents the magnitude of the gradient at each pixel.
where Ix is the image resulting from the convolution of kx and the image to be processed and Iy is the image
resulting from the convolution of ky and the image to be processed.
Another bit of resulting information which the Sobel edge detection produces is the direction of the edge
gradient. This can be of use for including or excluding edge elements and is described by the following
equation.
N.B. Remember to flip these kernels before applying them to your image.
9
D Gaussian Filtering
A Gaussian filter is a good way to remove noise and thereby avoiding potentially false edge detection. For
this application we will arbitrarily choose the following to be the kernel for the Gaussian filtering. - 4 7 4 1
- 16 26 16 4
- 26 41 26 7
- 16 26 16 4
- 4 7 4 1
(17)
E Non-maximum Suppression
This is one technique used to thin edges. Once edge detection has been performed we can apply nonmaximum
suppression which thins the edges and thereby only the thicker more distinct edges remain while
the thin non-important ones disappear.
To implement this we use the results from an edge detection algorithm in our case Sobel edge detection
(c.f., Appendix C) and using them in the following procedure. - Create a new image I
t
f which is constructed by applying a threshold to each pixel in If
. That is, for the
temperature value of each pixel uniquely identified by a row r and a column c, and denoted as If(r, c)
if the value of this pixel is greater than or equal to our selected threshold value t then we keep its value
by storing it in our new image at location I
otherwise we set the element to zero. For example, if our
starting image denoted by If
is as shown below - 2 3 4 5 6
- 10 55 119 155 91 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 61 124 149 86 10
- 10 64 127 146 83 10
and the threshold t = 124 then our new threshold image, denoted by I
, would result as follows. - 2 3 4 5 6
- 0 0 0 155 0 0
- 0 0 124 149 0 0
- 0 0 124 149 0 0
- 0 0 124 149 0 0
- 0 0 124 149 0 0
- 0 0 127 146 0 0
- Take the image (array) of gradient directions Θ as determined by applying, for example, Sobel edge
detection and round each element to the nearest value from the set (0o
, 45o
, 90o and 135o
) to create a
new gradient direction image Θ. ˉ - Process each non-zero pixel of I
t
F by looking at its gradient direction i.e., the value of Θ and checking ˉ
the two adjacent pixels along the direction of the gradient. If either of those two pixels have a gradient
magnitude greater than the current test pixel then set the test pixel to 0 otherwise leave it alone.
Repeat for all remaining pixels. (N.B. If one of the pixels adjacent to the test pixel is outside the
image we assume it magnitude is 0).
The entire process results in a new image containing only the strongest magnitude pixels.
10
F Double-Thresholding and Edge Tracking
This step aims to strengthen the remaining edges after a considerable amount of processing. This technique
can be summarised as follows. - Select a low magnitude threshold tl and a high magnitude threshold th such that tl < th.
- Create a new image T which results from taking an image and applying the thresholds such that if the
image pixel magnitude is less than tl
it is set to zero. If the magnitude is greater than th it is set to
maximum. And if the magnitude is between tl and th then we set its magnitude to half maximum. - Take T and compare each of the eight magnitude pixels. If a strong pixel is present in the set of 8 then
make this pixel strong otherwise set it to zero.
G Canny Edge Detector
This edge detector gives good detection for real images. The steps required to implement this detector are
as follows: - Remove noise by applying a Gaussian filter (see D).
- Determine the intensity gradients for the image by applying Sobel edge detection (see C).
- Remove spurious edge detection by applying non-maximum suppression (see E).
- Detect potential edges by applying double-threshold (see F).
- Apply hysteresis to track edges (see F).
H Run Length Encoding
Run length encoding (RLE) is a technique for compressing data. It is a simple approach which works very
well when the data contains many similar values. Conversely if each value in a data set is different then no
compression will result. The format of the compressed data is as follows. There is a header where the first - bytes represent the width (2 bytes) and height (2 bytes) of the array, e.g., 80 60.
After the header, each element of the compressed array is represented by two sequential 2 byte entries
(16 bits each). The first entry is the number of repetitions and the second is the actual value that is repeated
(in this specific case it is a 14 bit integer representing the thermal intensity).
For example, a compressed file that has the following values - 2 3 4 5 6
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
- 10 10 200 200 10 10
would be compressed to the RLE file - 6 2 10 2 200 4
- 2 200 4 10 2 200
- 10 2 200 4 10 2
- 4 10 2 200 2 10.
Note that the above RLE encoded file would not appear as a table. It would appear as one long line. This
is only done for visual clarity. Similarly the RLE input would be converted into a single line of values for
the encoding process hence the need to specify a header to describe the width and height of the compressed