Thesis

Outline of Thesis
For
Comparative Framework Review Of Image
Processing Toolkit

GCIS 546 Managing Information Organization

Version 1.0

Prepared by Priyanka R Desai

Nov 18, 2008

GCIS 546 Managing Information Organization

-1-
Thesis

Table of Contents
Table of Contents.............................................................................................2
1. Introduction ...............................................................................................3
2. Background.................................................................................................. 3
3. Methods for Comparing the Software...........................................................7
4. Reference applications Implementation.......................................................8
5. Results........................................................................................................10
6.Conclusion...................................................................................................12
7.Appendix..................................................................................................... 12

GCIS 546 Managing Information Organization

-2-
 Thesis

1. Introduction

Image processing is nothing but signal processing, for which the input is either an image
or video and output can be an image, video or set of characterstics from the image. The
typical operations within image processing are,

 Geometric transformation of image - scale, rotate, translate

 Color correction – brightness, contrast variation, color space modification
 Compositing – combining two or more images
 Image editing – to enhance the quality of image
 Image segmentation – used to partition the image in smaller segments
 Image recognition – patterns in the image can be recognized

With the advancement of computer technology, image processing has gained

momentum and aside from the typical operations there are lots of different things that
can be done with imaging now and is being used in varying fields of science &
technology.

 Computer vision
 Face detection
 Feature detection
 Lane departure warning system
 Non-photorealistic rendering
 Medical image processing
 Microscope image processing
 Morphological image processing
 Remote sensing

Computer vision basically is concerned with the theory of buidling the systems that
obtain information from images. This information can be used in industrial robots, people
counting, computer human interface. Face detection is recognizing the human face
amidst all the background. This combined with pattern recognition can be used in
security systems. Lane departure system is a safety feature that most OEMs are using in
cars now. This warns the driver of lane departure unless he is turning (turn signal is on).
Non-photorealistic rendering is used in computer graphics for photorealism. Medical
imaging is the technique & processes to create images of human body or parts of human
body. This has found extensive application in clinical studies & diagnisis. Microscope
imaging is digital image processing of images obtained from microscopes. This helps to
read the images from microscope for better understanding of concerns. Remote sensing
is gathering information from a device which is not in physical contact with the object.
Some of the examples are satellite information, earth observation, ultrasound and
Magnetic Resonance Imaging (MRI). There is lot of growth potential for image procesing
and still a lot of software development needs to be done.

2. Background

GCIS 546 Managing Information Organization

-3-
Thesis

There are two major open-source toolkits available for image processing & visualization.
They are, ITK (Insight toolkit) and VTK (Visualization toolkit). The VTK provides a wide
variety of visualization algorithms including scalar, vector, tensor, texture, volumetric
methods and advanced modeling techniques such as implicit modeling, polygon
reduction, mesh smoothing, cutting, contouring, and Delaunay triangulation. In addition,
dozens of imaging algorithms have been directly integrated to allow the user to mix 2D
imaging / 3D graphics algorithms and data. On the other hand ITK provides, registration
& segmentation algorithms.

The current comparison study has two criteria and the frameworks are chosen based on
meeting these two criteria.
1. The framework has to be freely available to benefit people of all budgets. This
helps all three categories of professional work areas, which are, Education,
Scientific & Engineering Computation and Commercial product development.
2. The framework has to have available interface to ITK and VTK.
There are five major image processing & visualization softwares available that meet
our requirement. Five softwares are, SCIRun, VolView, MeVisLab, and Medical
Interaction Toolkit (MITK), Matlab. All these are available for free and they all use
ITK C++ library. We are focusing on benefits & limitations of each to help the user
and the application developer in making the decision for using the correct framework
for their application.

a. SCIRun b. VolView c. MeVisLab d. MITK

Figure 1. Demonstration images of the compared visualization frameworks

SCIRun

SCIRun has integrated development environment (IDE) for visual data-flow programming
that connects a series of data processing modules. SCIRun includes more than 600
modules of varying degrees of complexity for visualization of scalars, tensor fields, 3D +
time, simulation using Finite Element method. It can be interfaced to Matlab, as well as a
number of utility programs that convert data stored in non-SCIRun formats. New
modules can be built with a wizard and can call custom C++ code. SCIRun also allows
one to create a “PowerApp”—an end-user application based on an existing data flow
network, but with fixed module connections and a streamlined workflow.

GCIS 546 Managing Information Organization

-4-
Thesis

Power-Apps –End User Application Area

BioImage - Application for displaying and analyzing biomedical images

BioTensor - Application for displaying and analyzing tensor data
BioFEM - Application for calculating electric fields in a finite element mesh

Figure 3. Bio-Tensor Power-Apps

An important application of diffusion tensor imaging is determining the connective
structure of central nervous tissue; white matter within the brain is more anisotropic
than gray matter on its cortical surface. Traces the connectivity along white matter
tracts by forming paths that follow the tensor principal eigenvector or use the tensor-
line algorithm.

Medical Imaging Interaction Toolkit – MITK

The Medical Imaging Interaction Toolkit, MITK is a C++ library for the development of
medical imaging software, but it can handle non-medical data as well. All ITK and VTK
functionalities are available in MITK. In addition, MITK can display synchronized
multiviewer layouts, and curved reformations (CR) of data, such as 3D + time. MITK also
promotes high-level interactions, such as construction and modification of data objects,
can display data with multiple states, and supports undo/redo. As a toolkit, MITK can be
added by developers to existing software and also allows the construction of end-user
applications specifically tailored for an image analysis task, providing only required
features. Programs are written in C using Microsoft Visual Studio or C/C++ compiler.
End User Applications – Medical Users

GCIS 546 Managing Information Organization

-5-
Thesis

Figure 6. End User Medical application for MITK

2.3 Volview
VolView is an intuitive, interactive system for volume visualization that
allows researchers to quickly explore and analyze complex 3D medical or scientific
data on a standard PC. Users can easily generate informative images for further
analysis. 3D tools include volume rendering, contouring and cropping Segmentation
can be performed, and resulting structures can be analyzed using distance, angle
and surface area and volume measurement tools.
Data exploration is enhanced through tools such as filtering,
contours, measurements, histograms, animation, and annotation. Advanced users
can extend VolView data processing using the simple plug-in API. Researchers can
collaborate with distant colleagues by trading control between two synchronized
sessions. VolView is an end-user application for volume visualization designed for
the exploration of 3D medical or scientific images. Through efficiently accessible
tools data can be visualized through synchronized multiviewer layouts, annotations,
and gradient magnitude transfer functions and analyzed by measurements and
histograms. No programming skills are required to use the standard functionality of
VolView, as all is available through menus and GUI panels to set algorithm
parameters. Currently, there are 37 ITK and VTK filters with customized GUI panels
available. VolView also offers a plug-in interface that programmers can use to make
their own image processing filters available inside the VolView filter menu.
Development environment is C/C++.

MeVislab
MeVisLab, the successor of the image-processing environment ILAB4,
represents a platform for image processing research and development with a focus
on medical imaging. Beside general image processing algorithms and visualization
tools, MeVisLab includes advanced medical imaging modules for segmentation,
registration, volumetric and quantitative morphological and functional analysis.
MeVisLab is an IDE for visual data-flow programming with more than 500 modules of
varying degrees of complexity. Modules can be combined into data processing
networks in which image data is processed in pages on an as-needed basis using
priority-controlled page caching. Data can be 3D + time and visualization is based on
Open Inventor scene graphs and provides modules for, 2D transfer functions, iso-
surfaces, slab rendering, MPR, CR, and overlay display of metadata or ROI (Region
of Interest). The framework is designed for integration of custom C++ code and
macro-modules. An end-user application typically for health professionals can be
built based on a network, just as a SCIRun PowerApp. Dynamic functionality can be
added using Java Script.

2.5 Matlab
Matlab image processing toolbox is not just restricted to medical applications. It
includes advance-imaging modules for segmentation, registration, and quantitative
morphology. Matlab image processing toolbox has specialized features for image
deblurring, enhancement, spatial transformation and measuring image features. It
also has filtering, contours, measurements, histograms, animation, and annotation.

GCIS 546 Managing Information Organization

-6-
Thesis

Matlab has separate video processing toolbox for 3D+time real-time applications.
Matlab has separate specialized module for image acquisition. Matlab applications
can be integrated with various C/C++, excel, java applications. Standalone
executable can be generated to run matlab applications on non-matlab installed
systems. In matlab image-processing functions can be converted to standard C/C++
library dll files that can be included as normal dll file in any other applications
development for using functions available in dll file. Matlab provides GUIDE that is
used to prepare Graphical user interface for image processing applications.

3. Methods for Comparing the Software

Different evaluation criteria were used to highlight the differences between different
frameworks. In this case, we have used a reference example (Spinal Cord MRI) to
compare the different frameworks.

3.1 Functionality and application criteria

• Color and Opacity Transfer Function Volume Rendering
Displays 2D projection on a 3D volume with RGBA. This is mostly used in CT of
fractured bones.
• Gradient function transfer volume rendering
Enhances regular volume rendering by suppressing samples along the projection
line. This is mostly used in applications like tumors and airport security.
• Multidimensional function volume rendering
Displays a histogram of two arbitrary scalar values derived from the same volume
data. User can set opacity and color that are then used for volume rendering.
Mostly used present different regions of tooth and explosion simulation results.
• Tagged Volume rendering
Uses different transfer functions, shading effects for each uniquely tagged region.
It can be used to see pistons inside an engine block or to display inside human
organs.
• Finite element analysis
The volume is divided into tiny elements with varying grid. Used in EEG and MEG
measurements.
• Partial differential equation solving
Computes a numerically approximated solution to one or multiple coupled partial
differential equations. Can be used to compute steady state frequency selections
during laser light amplification.
• Curved reformation
Data or image is visualized as 2D map or 3D textured surface. Curved
reformations can be bent along different axes. An example of the application is
the entire spine in a single curved view or maps of earth.
• 3D reformed mesh Segmentation

GCIS 546 Managing Information Organization

-7-
Thesis

Uses an interface to verify and steer the mesh deformation process. This
functionality can be used in brain tumor segmentation, as well as segmentation of
cells.
• 3D Sampled data exploration
Implies not knowing which parameters are optimal to visualize the data. Used in
3D ultrasound.
• Interactive Segmentation parameter finding
Displays on a GUI all relevant segmentation parameters for each supported
segmentation filter/algorithm as well as the resulting segmentation output. Used
in MRI scanner.
• Interactive prototyping of research algorithm
Combines previously developed algorithm components in new configurations with
customized parameter settings. Used in removing red eye effect on digital
camera images.
• Analysis of 3D+ Time Volume Data
Imports 4D volume data, processes and visualizes profile curves along time for
each 3D position and creates parametric maps that show classifications of profile
curves.

4. Reference applications Implementation

The reference application implemented was found on the website of respective

software site which includes typical features required to combined visualization of
volume and geometry data. Specifically, the application semi automatically determines
the centerline of the spinal cord. Data processing is performed using ITK. Screenshots
for the application on each framework were shown for comparison. Here in the program,
input image is MRI scan of back od human body and output image is spinal cord image.
Here the objective is to find the spinal cord in the image. The user clicks on the spinal
cord region found in the image, and provides the seed points which is used for edge
detection and segmentation. When connected spinal cord edge is detected, closed
region is formed using 8- connectivity function. This closed region is enhanced and
background is removed from rest of the image. Spinal cord axis is calculated and straight
line is drawn alon the axis to connect the seed points.

4.1 SCIRun
Explaining the development and results obtained for the project developed in
SCIRun is shown in Figs. 10, 11, 12, and 13. Project setup includes ITK classes present
in modules. Modules were selected from a context menu and then connected by drag
and drop of visual pipes between TCL/TK GUI components representing the modules.
Each module has its own pop-up GUI to set parameters which required setting for the
application. Figure 10 shows the development environment of SCIRun. Intermediate
results of the program are displayed of original MRI data (gray-scale) overlaid with the

GCIS 546 Managing Information Organization

-8-
Thesis

segmentation mask in Figure 11. Figure 12 is one of the intermediate results where
noise and background has to be removed from the spinal cord’s region of interest. Final
results is shown in Figure 13 where just the identified spinal cord is displayed.

4.2 MITK

Explaining the development and result for the project developed in MITIK. Figures.14,
15, 16, and 17. The project setup includes using existing functionality. Here project is
compiled and executed the application after each ITK class was added to the MITK
computation pipeline. The test runs were performed with a single image 2D data set. The
resulting centerline was represented as tubes and spheres at subvoxel accurate
locations in Figure 17. Through the display functionality, the user can set those
properties, such as visibility, color, size of the points or tubes, volume rendering on/off,
and transfer functions.

4.3 Volview

Explaining the development and results for the project developed in VolView are shown
in (Figs 18, and 19). The project setup includes using existing plug-in from ITK classes.
Adding seed points was already an existing feature of VolView. ITK filter are set from the
menu and moving sliders to set the input parameters. Input to the program is 3D data
sets, as VolView is designed for volumes, not for 2D images. Display of the input and
computed data sets was presented automatically as the current result became available
after each interactive step. The final axis of spinal cord is shown in Figure 20.

4.5 MeVislab
Explaining the development and results for the project developed in MeVislab are shown
in Figure. 21, 22, 23, and 24. The project setup included ITK classes strictly complying
with ITK coding style guidelines. Adding seed points was done with a standard
MeVisLab module. Input to the program is 2D data. Module outputs were displayed in
multiple viewers that could be configured to synchronize their view position and
visualization parameters. End-user application which stores parameter defaults for 2D
and 3D was created using Java Script (Figs. 22 and 24).

4.6 Matlab
Explaining the development and result for the project developed in Matlab are shown in
Figure 25,26,27,29. The project includes IDE of video processing block-set as shown in
figure 25, which is used as visual programming tool. This video processing block-set has
integrated ITK and VTK that are the underlying programming language on which the
block-set is built. Input to the program is 2D data as shown in figure 26. Edge detection,
background subtraction is initial preprocessing for the image. Seed points are given as
input that are shown as red dots on figure 27, Axis of spine is shown in figure 28 and
finally spine is reconstructed by meshing as shown in figure 29.

GCIS 546 Managing Information Organization

-9-
Thesis

5. Results
Both application developer & application user evaluations are presented.

5.1 Application Developer-Oriented Evaluation

Table 1 summarizes the framework differences in application developer functionalities.

Application Developer SCIRun MITK VolView MeVisLab Matlab

Criteria

Framework Features

Visual programming (fast prototyping, +1 -1 0 +1 +1

parameter w/o recompile)

The ease of working with ITK and VTK 0 +1 -1 0 +1

Problem solving support

Runtime error tracking. 0 +1 -1 +1 +1

License

+: Open source ITK, 0: free run time and +1 +1 0 0 +1

plug-in API, -: only commercial

Operating System Support

+: Three or more out of windows, Linux, +1 +1 0 0 +1

Mac-OS, IRIX, 0: two, -: one

Non-GUI Command Line Applications -1 0 -1 0 +1

Total 2 5 -3 2 6

Table 1: Comparison of all frameworks by application developer evaluation criteria

GCIS 546 Managing Information Organization

- 10 -
Thesis

5.2 Application User –Oriented Evaluation

Table 2 summarizes the framework differences in functionalities that are of importance to

application users.
Application User Evaluation Criteria SCIR MITK VolView MeVisLab MatLab
un

Framework Features

Number of algorithms with user interfaces. +1 -1 0 +1 +1

+: Many, 0:fewer, -1:least

Already integrated scientific packages & +1 0 0 +1 +1

tools, +1:3 or more of ITK, VTK

Virtual memory. +1:2GB, 0:1GB, -1:512KB -1 0 0 -1 +1

Undo/redo of data processing. +1:unlimited +1 -1 0 +1 +1

undo or milestones, 0:single undo, -1: no
undo

Undo/redo of algorithm module 0 0 -1 +1 +1

configuration. +1:unlimited undo, 0:
unlimited milestones, -1: no undo

GUI Features

Availability of widgets for the application +1 +1 0 0 +1

GUI. +1: full widget library, 0: reduced set,
-1:none

Functionalities

+1:significant help, 0:possible, -1:outside

scope of framework

Color/opacity Transfer function rendering +1 +1 +1 +1 +1

Gradient Magnitude TF volume rendering +1 -1 +1 +1 +1

Multidimensional TF volume rendering +1 -1 -1 +1 +1
Tagged volume rendering 0 -1 -1 +1 +1

Finite Element Analysis +1 -1 -1 -1 +1

Partial differential equation solving +1 -1 -1 -1 +1

Curved Reformations -1 +1 -1 +1 +1

3D Deformable Mesh segmentation -1 +1 -1 -1 +1

Data Exploration 0 0 +1 0 +1

GCIS 546 Managing Information Organization

- 11 -
Thesis

Interactive Segmentation Parameter finding +1 0 +1 +1

+1

Interactive Prototyping of Research +1 -1 -1 +1 +1

algorithm
-1 0 -1 +1 +1
Analysis of Time-resolved Volume data

Total 7 -4 -5 8 18

6.Conclusion
Image processing softwares are crucial for image processing developers as well as
users. There isn’t a specific survey done for developers to choose which image
processing software is best suited for their application. As different mathematical
algorithms applied to images gives different results for same application and accuracy of
results change accordingly. This paper is focused to compare image-processing
softwares such as SCIRun, MeVislab, Matlab, MITK, and VolView. Matlab scored very
high in our comparison tests and we conclude that Matlab image processing toolkit is
better among the five softwares compared. Matlab has all the functionality and its not
restricted to medical application. Matlab image processing can be applied to non-medical
applications as well. Matlab has separate specialized image acquisition toolbox as well
as video processing block-set, which makes it unique as whole computer vision project
can be set up easily.

7.Appendix
1. MITK:
The Functionality Generator is a tool to simplify the process of creating
your own MITK functionality and integrating it into the SampleApp.
Functionality contains the application main widget. The functionality has to set up the
main widget according to its needs when it is activated or reverse the settings when it
is deactivated.

GCIS 546 Managing Information Organization

- 12 -
Thesis

Figure 5. Functionality Generator for MITK

Cmake is used to generate a Makefile on UNIX platform and Visual Studio

workspace on windows respectively for MITK.

Figure 4. Make file generation with MITK

2. SCIRun can also be used as Problem Solving Environment (PSE) for modeling,
simulation and visualization of scientific problems. Each specific PSE, such as BioPSE,
is a package within SCIRun. PSEs use and build upon data types, algorithms, and
modules provided by the SCIRun framework. PSEs provide application specific data
types, algorithms and modules.

GCIS 546 Managing Information Organization

- 13 -
Thesis

Figure 2. SCIRun/BioPSE – package

3.

ITK-SNAP
A tool for viewing medical images and for delineating and extracting anatomical
structures. ITK-SNAP can be used in two different modes: manual segmentation and
semi-automatic segmentation. The manual mode is used for segmentation using
hand contouring and for cleaning up the results of automatic segmentation. In the
semi-automatic mode, a powerful level set segmentation algorithm is used to
segment anatomical structures in three dimensions. This algorithm requires some
guidance from the user, and ITK-SNAP provides an easy interface to provide such
guidance.

Figure 7. Human ankle using ITK-SNAP of VolView

4.

GCIS 546 Managing Information Organization

- 14 -
Thesis

Figure 8. Development Environment in MeVisLab

Figure 9. High Quality Volume Rendering by MeVisLab

GCIS 546 Managing Information Organization

- 15 -
Thesis

Figure 10. SCIRun: Data-flow network, subnetwork, and module parameter

Figure 11. SCIRun: Display of original MRI data (gray-scale) overlaid with the
segmentation mask (brown blend) and overlaid with the distance transform data (red-
green-blue colored grid, voxel values as text, and voxels coordinates as text).

Figure 12. SCIRun: Single slice 2D input data, seed points, spinal cord segmentation,
and centerline on single view.

GCIS 546 Managing Information Organization

- 16 -
Thesis

Figure 13. SCIRun: 3D spinal cord data, seed points, segmentation, and
centerline, as well as object browser, 1D volume rendering transfer
function editor, and color map selector.

Figure 14. MITK: Single Slice 2D input data, seed points, spinal cord,
segmentation, and centerline on single view.

Figure 15. MITK: 3D data, seed points, segmentation, and centerline on

correlated 2D and 3D views with input GUI.

GCIS 546 Managing Information Organization

- 17 -
Thesis

Figure 16. MITK: 3D data, seed points, two centerlines, large 3D view

Figure 17. MITK: Fig. 7 data and layout with curved reformation and plane

Figure 18. VolView: Speed image, four-quadrant view, and GUI panel for appearance
(transfer function editors).

GCIS 546 Managing Information Organization

- 18 -
Thesis

Figure 19. VolView: Segmentation views of the spinal cord segmentation with the
smoothed MRI, appearance GUI.

Figure 21. MeVisLab: Data-flow network and sub-network, the selected crop module is
highlighted and configured to automatically display input and output images and image
meta-information.

Figure 22. MeVisLab: Single Slice 2D input data, spinal cord segmentation, and
centerline, correlated 2D views in customized application.

GCIS 546 Managing Information Organization

- 19 -
Thesis

Figure 23. MeVisLab: GUI parameters edit boxes and multiple viewers for 2D data,
spinal cord segmentation, and centerline.

Figure 24. MeVisLab: Streamlined end-user application showing 3D data on correlated

3D view, cross-sectional view, and stretched curved reformat.

GCIS 546 Managing Information Organization

- 20 -
Thesis

Figure 25. Development IDE in Matlab

Figure 26.Input 2D Spine Image for Matlab program

GCIS 546 Managing Information Organization

- 21 -
Thesis

Figure 27. Edge detection with seed points for Matlab

Figure 28. Matlab: Original 3D data with seed points

GCIS 546 Managing Information Organization

- 22 -
Thesis

Figure 29. Reconstruction of Spine image in Matlab

GCIS 546 Managing Information Organization

- 23 -