Anda di halaman 1dari 31

## Edemir Ferreira de Andrade Junior

edemir.matcomp@gmail.com

DCC
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Image processing vs. Image Analysis

1 Image processing: Methods where input and output are images, e.g.
Point-wise operations
Local neighborhood operations
Fourier transform.

2 Image analysis: Methods where the input is an image, but the output are
attributes extracted from input, e.g.
Image segmentation
2D and 3D object recognition
Optical flow

2 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

What is segmentation?
Dividing the image into different regions.
Separating objects from background and giving them individual ID numbers
(labels).

(a) (b)

3 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Wikipedia on segmentation

## 1 ”In computer vision, Segmentation is the process of partitioning a digital image

into multiple segments”

## 2 ”More precisely, image segmentation is the process of assigning a label to every

pixel in an image such that pixels with the same label share certain visual
characteristics.”

3 Each of the pixels in a region are similar with respect to some characteristic or
computed property, such as color, intensity, or texture. Adjacent regions are
significantly different with respect to the same characteristic(s).”

4 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Why segmentation?

## Accurate segmentation of objects of interest in an image greatly facilitates further

analysis of these objects. For example, it allows us to:

## 1 Count the number of objects of a certain type.

2 Measure geometric properties (e.g., area, perimeter) of objects in the image.
3 Study properties of an individual object (intensity, texture, etc.)

5 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Segmentation

## THERE IS NO UNIVERSAL SOLUTION! =O

6 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Targeted Segmentation
1 Segmentation is an ill-posed problem...

## Figura: What is a correct segmentation of this image?

7 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Targeted Segmentation
1 ...unless we specify a segmentation target.

(a) Segment the orange car from the (b) Segment all road signs from the
background? background?

8 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Targeted Segmentation

A segmentation can also be defined as a mapping from the set of pixels to some
application dependent target set, e.g.
1 Object, Background
2 Humans, Other objects
3 Healthy tissue, Tumors

## To perform accurate segmentation, we (or our algorithms) need to somehow

know how to differentiate between different elements of the target set.

9 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Segmentation

Segmentation algorithms are often based on one of the following two basic
properties of intensity values:

Similarity
Partitioning an image into regions that are similar according to a set of predefines
criteria.

Discontinuity
Detecting boundaries of regions based on local discontinuity in intensity.

10 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Four types of segmentation algorithms

1 Thresholding - Similarity
Based on pixel intensities (shape of histogram is often used for automation).
2 Mean Shift - Similarity
Aims to discover ”blobs”in a smooth density of samples.
3 Watershed segmentation - Discontinuity
Find regions corresponding to local minima in intensity.
4 Simple Linear Iterative Clustering (SLIC) - Similarity
Cluster pixels from a five-dimensional space into nearly uniform superpixels

11 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Mean Shift

Mean Shift is a powerful and versatile non parametric iterative algorithm that can
be used for lot of purposes like finding modes, clustering, segmentation etc.

Mean Shift was introduced in Fukunaga and Hostetler 1 and has been extended
to be applicable in other fields like Computer Vision.

1
Fukunaga, Keinosuke; Larry D. Hostetler (January 1975). ”The Estimation of the Gradient of a Density
Function, with Applications in Pattern Recognition”. IEEE Transactions on Information Theory. IEEE. 21 (1):
32?40. doi:10.1109/TIT.1975.1055330. Retrieved 2008-02-29.
12 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Mean shift considers feature space as a empirical probability density function.

If the input is a set of points then Mean shift considers them as sampled from the
underlying probability density function.

When dense regions (or clusters) are present in the feature space, then they
correspond to the mode (or local maxima) of the probability density function.

13 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Intuitive Idea of Mean Shift

For each data point, Mean shift associates it with the nearby peak of the
probability density function.
For each data point, Mean shift defines a window around it and computes the
mean of the data point.
Then it shifts the center of the window to the mean and repeats the algorithm till it
converges.
After each iteration, we can consider that the window shifts to a more denser
region of the dataset.

14 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Intuitive Idea of Mean Shift

(c)

15 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## At the high level, we can specify Mean Shift as follows :

1 Fix a window around each data point.
2 Compute the mean of data within the window.
3 Shift the window to the mean and repeat till convergence.

16 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Mean Shift

No assumptions on the shape or number of data clusters.
The procedure only has one parameter, the ”window size”.
Output doesn’t depend on initializations.
Robust to outliers

Disvantages:
Computationally expensive for large feature spaces.
Often slower than K-Means clustering
Have to choose kernel size in advance
Output depends on window size.

17 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Watershed
1 The watershed algorithm was introduced by H. Digabel and Christian Lantuéjoul
in 1978 and extended by Serge Beucher in 1982.

## Figura: Serge Beucher

18 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Think of the gray level image as a landscape.

Let water rise from the bottom of each valley (the water from the valley it given its
own label).
As soon as the water from two valleys meet, build a dam, or a watershed.
These watersheds will define the borders between different regions in the image.
The watershed algorithm can be used directly on the image, on an edge
enhanced image or on a distance transformed image.

19 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Watershed - Drop of Water Analogy

Think of the gray level image as a landscape.

## (a) Original Image (b) Landscape

A drop of water landing at any point in the landscape will flow down to a local
minimum in the landscape.
For any local minimum in the landscape, there is a set of points, called the
catchment basin, from which a drop of water will flow to that given minimum. 20 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Watershed - Drop of Water Analogy

A drop of water landing at any point in the landscape will flow down to a local
minimum in the landscape.

(c)
For any local minimum in the landscape, there is a set of points, called the
catchment basin, from which a drop of water will flow to that given minimum.
The boundaries between adjacent catchment basins form the watershed.

21 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Distance Transform

## (d) Original Image (e) Distance Transform

22 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Watershed - Example 1
Example of watershed on distance transformed image:

(f) Original Image (g) Local minima has been (h) Flooding starts from the
found, indicated by red dots minima

(i) High ridges prevents merging (j) Flooding reaches a point, (k) Output 23 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

Watershed - Example 2
Example of watershed on distance transformed image:

24 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Algorithm by F. Meyer (early 90’s)

Computing watersheds:
1 A set of markers, pixels where the flooding shall start, are chosen. Each is given
a different label.
2 The neighboring pixels of each marked area are inserted into a priority queue
with a priority level corresponding to the gray level of the pixel.
3 The pixel with the highest priority level is extracted from the priority queue. If the
neighbors of the extracted pixel that have already been labeled all have the same
label, then the pixel is labeled with their label. All non-marked neighbors that are
not yet in the priority queue are put into the priority queue.
4 Redo step 3 until the priority queue is empty.
The non-labeled pixels are the watershed lines.

25 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## In practice, this transform produces an important over-segmentation due to noise

or local irregularities in the gradient image.
A major enhancement of the watershed transformation consists in flooding the
topographic surface from a previously defined set of markers.
Doing so, we prevent any over-segmentation.

26 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Simple Linear Iterative Clustering

Simple Linear Iterative Clustering (SLIC) is the state of the art algorithm to
segment superpixels which doesn’t require much computational power.

This algorithm was developed at Image and Visual Representation Group (IVRG)
at EPFL in published paper2

2
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P. and Süsstrunk, S., 2012. SLIC superpixels
compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine
intelligence, 34(11), pp.2274-2282.
27 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

What is a superpixel?
A superpixel can be defined as a group of pixels which have similar
characteristics. It is generally color based segmentation
Superpixels can be very helpful for image segmentation.

There are many algorithms available to segment superpixels but SLIC is the a
state of the art with a low computational overhead.
28 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## Simple Linear Iterative Clustering

In brief, the algorithm clusters pixels in the combined five-dimensional color and
image plane space to efficiently generate compact, nearly uniform superpixels.

## But how it works?

29 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## How does SLIC work?

The approach is really simple actually. SLIC performs a local clustering of pixels
in 5-D space defined by the L, a, b values of the CIELAB colorspace and x, y
coordinates of the pixels.

## It has a different distance measurement which enables compactness and

regularity in the superpixel shapes, and can be used on grayscale images as well
as color images.

## A variable m is introduced in Ds allowing us to control the compactness of

superpixel. The greater the value of m, the more spatial proximity is emphasized
and the more compact the cluster

30 / 31
DCC029/868 - Processamento Digital de Imagens
Processsamento de Imagem vs Analise de Imagem

## How does SLIC work?

It begins by sampling K regularly spaced cluster centers and moving them to seed
locations corresponding to the lowest gradient position in a 3 x 3 neighborhood.

Each pixel in the image is associated with the nearest cluster center whose
search area overlaps this pixel

After all the pixels are associated with the nearest cluster center, a new center is
computed as the average labxy vector of all the pixels belonging to the cluster.

31 / 31