Anda di halaman 1dari 10

Design of an industrial machine vision system for inspection

and assembly using soft computing

1st year report submitted in the partial fulfillment of the requirements for the
degree of

DOCTOR OF PHILOSOPHY

By

Bondada Venkatasainath
(15ME92R09)

Under the supervision of


Prof. D. K. Pratihar & Prof. C.S.Kumar

Department of Mechanical Engineering


Indian Institute of Technology, Kharagpur
June, 2016
Design of an industrial machine vision system for inspection
and assembly using soft computing

1. Introduction

Nowadays, customer wishes push demands on variety of individual product features, and shorter
delivery time in manufacturing industry [1]. With a high degree of complexity of manufacturing processes,
the automation systems have been applied to production and final assembly systems. In the high mix
production, product models may have various sizes, shapes, and orientations which require a higher
flexibility of the automatic assembly process. When the product models are changed in the conventional
assembly process, the production line needs to be stopped and then new configuration and commands
of the automatic assembly system need to be set up.

This conventional assembly process can increase the production lead time, whereas it can
decrease the systems efficiency. Consequently, the production costs are raised by investing in labor cost,
machine and equipment cost. To improve the conventional automatic assembly system, the flexible
automatic systems have been utilized in the high mix assembly process. This flexible automatic system
can modify its working pattern and automatically responds to the new models. In this case, the production
line does not need to be stopped remarkably improving the assembly process performance.

One way to accomplish the flexible automatic system is to integrate machine vision systems with
the automation systems. The computer vision techniques have been used to provide product data which
assists decision-making of the production systems [2]. In cutting tool process using CNC machine, the
vision sensor have been used to monitor cutting tool conditions, including tool wear, and surface texture.
In this case, the information on tool wear state obtained from the vision sensor was used to estimate wear
parameters which were used as feedback control for CNC controllers [3].

An automated vision system has also been used to inspect defects on printed circuit boards using
reference comparison approach. This automated visual inspection system was considered to be efficient
for defect detection and defect classification [4]. In the automatic assembly process, the machine vision
system is also feasible to inspect the characteristics, size, shape, orientation, and defects of the product
models. This inspection assists the automatic assembly system to distinguish the models and respond to
those models correctly. The vision system basically based on image classification. First, the object image
is captured using camera. Next, the search area of the image is specified to set the environment for image
processing.
In the image processing step, the classification is identified by comparing the significant features
of the captured image to those of the standard image. Two major classifiers are generally used for the
image processing: object classifier, and color classifier. The object classifier identifies the object based
on its shape while the color classifier distinguishes the object based on its color [5].

Meanwhile, industrial robots have been also implemented in the automatic assembly process to
reduce human errors, lead time, and labor cost. The robots can achieve better performance using novel
control methods. Chen and Liu have remarkably succeeded in implementing the robust impedance control
algorithm with a selective compliance articulated robot arm (SCARA) to perform a printed circuit board
(PCB) assembly [6].

The requirements for the design and development of a successful machine vision system vary
depending on the application domain and are related to the tasks to be accomplished, environment,
speed, etc. For example, in machine vision inspection applications, the system must be able to
differentiate between acceptable and unacceptable variations or defects in products, while in other
applications, the system must enable users to solve guidance and alignment tasks or, measurement and
assembly verification tasks.

For the system to be reliable, it must reduce escape rates (i.e. non-accepted cases reported as
accepted) and false alarms (i.e. accepted cases reported as non-accepted) as much as possible. It is a
responsibility of the processing and classification units to maintain system reliability, but the effectiveness
of classification depends also on the quality of the acquired images. An industrial vision system must also
be robust. Thus, it should adapt itself automatically and achieve consistently high performance despite
irregularities in illumination, marking or background conditions and, accommodate uncertainties in
angles, positions, etc. Robust performance is difficult to achieve. High recognition and classification rates
are obtained only under certain conditions of good lighting and low noise. Finally, an industrial vision
system must be fast and cost efficient. In this survey, we emphasize the important attributes of an
industrial machine vision inspection system such as, flexibility, efficiency in performance, speed and cost,
reliability and robustness. In order to design a system that maintains these attributes it is important to
clearly define its required outputs and the available inputs.

Typically, an industrial inspection system computes information from raw images according to the
following sequence of steps:

1. Image acquisition:
Images containing the required information are acquired in digital form through cameras, digitizers, etc.

2. Image processing:
Once images have been acquired, they are filtered to remove background noise or unwanted
reflections from the illumination system. Image restoration may also be applied to improve image quality
by correcting geometric distortions introduced by the acquisition system (e.g. the camera).

3. Feature extraction:
A set of known features, characteristic for the application domain, is computed, probably with
some consideration for non-overlapping or uncorrelated features, so that better classification can be
achieved. Examples of such features include size, position, contour measurement via edge detection and
linking, as well as and texture measurements on regions. Such features can be computed and analyzed by
statistical or other computing techniques (e.g. neural networks (NNs) or fuzzy systems (FSs)). The set of
computed features forms the description of the input image.

4. Decision-making:
Combining the feature variables into a smaller set of new feature variables reduces the number
of features. While the number of initial features may be large, the underlying dimensionality of the data,
or the intrinsic dimensionality, may be quite small. The first step in decision making attempts to reduce
the dimensionality of the feature space to the intrinsic dimensionality of the problem. The reduced feature
set is processed further as to reach a decision. This decision, as well as the types of features and
measurements (the image descriptions) computed, depends on the application. For example, in the case
of visual inspection during production the system decides if the produced parts meet some quality
standards by matching a computed description with some known model of the image (region or object)
to be recognized. The decision (e.g. model matching) may involve processing with thresholds, statistical
or soft classification.

2. Literature review

This review includes image processing and analysis tools, as well as, tools based on NNs, fuzzy
logic and GAs.

2.1. Image processing and analysis tools


Image processing is usually performed within rectangles, circles or along lines and arcs. Image
processing operators include filtering (e.g. smoothing, sharpening), edge detection, thresholding,
morphological operations, etc. Such operations can be used to improve image quality (e.g. remove noise,
improve contrast) and to enhance or separate certain image features (e.g. regions, edges) from the
background. Image processing operations transform an input image to another image having the desired
characteristics.

Image analysis transforms images to measurements. In particular, image analysis is related to the
extraction and measurement of certain image features (e.g. lines, and corners) and transforms these
image features to numbers, vectors, character strings, etc. For example, lines, regions, characters, holes,
rips, tears can be gauged or counted. Image analysis involves feature extraction operations (e.g. Hough
transform for line and curve detection) in conjunction with operations that measure average light
intensity, texture, and shape characteristics such as Fourier descriptors, moments, edge thinning, edge
connectivity and linking, etc. The ultimate goal of image analysis is geared towards pattern recognition,
i.e. the extraction of features that can be used by classifiers to recognize or classify objects.
An image processing environment to be suitable for industrial inspection, must (at least) contain
algorithms for edge and line detection, image enhancement, illumination correction, geometry
transforms, ROI selection, object recognition, feature selection and classification. In terms of combined
software and hardware, there are at least four alternatives: (I) IM-PCI with IPL of Sherloc32/MVTools, (II)
MaxPCI with PC Image Flow or WiT and (III) Matrox Genesis with Matrox Imaging Library and (IV) Philips
Trimedia VLIW processor board with Rhapsody.

2.2. Neural networks (NN)


NNs are being successfully applied across a wide range of application domains in business,
medicine, geology and physics to solve problems of prediction, classification and control. NNs are
composed of a number of similar elementary processing units (neurons) connected together into a
network [7]. Neurons are arranged in layers with the input data initializing the processing at the input
layer. The processed data of each layer passes through the network towards the output layer. NNs adapt
the weights of their neurons during a training period based on examples, often with a known desired
solution (supervised training). After sufficient training, the NN is able to relate the problem data to the
appropriate solution spaces, i.e. generate input/output relations, thus offering a viable solution to a new
problem through examples [8]. They are capable of handling a variety of image classification tasks in
industrial vision environments, ranging from simple gauging to advanced classification problems, such as
fault detection, optical character recognition, operation prediction, engine monitoring and control, etc.
They can be used either as standalone techniques or in conjunction with other methods (e.g. solder joint
inspection). NNs have been applied in all classes of quality inspection, namely dimensional quality, surface
quality, structural quality and operational quality. They are applicable in almost every situation where a
relationship between input and output parameters exists, even in cases where this relationship is very
complex and cannot be expressed or handled by mathematical or other modelling means.

2.3. Fuzzy systems (FS) and neuro-fuzzy systems (NFS)

A variety of industrial vision applications of diverse nature have been benefited by the use of FSs
and neuro- fuzzy systems (NFSs). Examples of such applications are cork quality inspection [9],
identification of mechanical component dimensional tolerances [10], IC product quality control [11] etc.
Fuzzy sets are based on decisions on linguistic variables, which get linguistic values described by fuzzy
sets, called membership functions [12]. Their basic Industrial processing elements are fuzzy sets instead
of numerical values. A fuzzy set can be considered as an extension of a classical (crisp) set in the form that
a crisp set permits only full membership or no membership, whereas a fuzzy set permits partial
membership with a certain degree. For example a dimensional tolerance in length can define fuzzy sets
small, medium and large with respect to its values, as shown in the below Figure. :
Classical decision making systems usually try to avoid vague, imprecise or uncertain information.
FSs on the other hand, deliberately make use of this information through the membership functions that
describe the degree to which a measurement belongs to certain sets or classes. The membership function
is the essential component of a fuzzy set. Thus, the operations intersection, union and complement of
fuzzy sets are defined via the membership functions of the sets involved. Input values are matched with
the preconditions of ifthen rules on fuzzy sets, describing the systems behavior. This kind of structured
knowledge provides the system with a rule-based processing (or rule- based reasoning) mechanism. This
mechanism supports independent rules (i.e. changes in one rule do not effect the result of other rules.
FSs and NNs differ mainly on the way they map inputs to outputs, the way they store information or make
inference steps.

NFSs form a special category of systems that emerged from the integration of FSs and NNs. There
are two major subcategories characterizing this integration namely: neural network fuzzy systems
(NNFSs) incorporating FSs represented on a NN topology and fuzzy neural networks (FNNs) which
include neural structures with a number of fuzzyfied parts [9]. NNFSs aim at providing FSs with automatic
tuning methods typical to NNs but, without altering their functionality. NNs learning ability can be used
to build membership functions and rules to encode systems behavior. The learning techniques employed
are mainly based on multilayer feed forward networks with the back-propagation algorithm. FSs offer
their users better in- sight of neural black box structures, by encoding structured information in the form
of rules and by offering tools for exploring this knowledge.

FNNs retain the basic properties and architectures of NNs and simply fuzzify some of their
elements. The system obtains increased flexibility in storing, recalling and associating information. Not
only binary but also continues values can be given as inputs, which highly increases system robustness.
Usually, fuzzified com- ponents result in higher training speed of NNs. Fuzzy Logic also makes neural
models understandable and this increases users flexibility.
There exist a large variety of Neuro-Fuzzy topologies in the literature, such as the Fuzzy ARTMAP,
the MCFC for speech recognition,the ASAFES2 network. The most popular topology though is the ANFIS
[13], which follows the general structure of Neural-Fuzzy systems and moreover has an extra layer to
perform normalization of rules firing strengths.

Recently, GAs have also been employed to deal with certain tasks ns certain application domains.
For example, an application of GAs in industrial vision is pattern detection, which defers from pattern
matching in the sense that the item sought is not known in advance. The use of GAs has also been reported
for the classification of objects, the detection of circular objects, image segmentation [40] as well as for
the minimization of trim loses in cutting two-dimensional stock sheets , etc. Sugal 2.1 (TRAJAN Software
Ltd) and the Generator (NEW LIGHT INDUSTRIES Ltd) are the two packages that can simulate GAs for
various purposes and provide complete control over these algorithms. The main difference between them
is that, Generator is designed to interact with Excel while, Sugal is autonomous.

3. Motivation:

In order to describe the applications of machine vision systems; four categories of the visual
inspection, process control, parts identification, and robotic guidance and control mechanisms are
considered. In this field, the most significant task of the machine is for the automated visual inspections
(AVIs). The main concern of using machine is to recognize that the part is well made according to the
specified qualifications. AVI and parts identification do not contribute a significant role in the flexibility in
manufacturing, however have considerable role in the automation task. On the other hand, vision systems
in the process control and robotic guidance can play important role in achieving more flexibility in
manufacturing.

3.1. Automated visual inspection


The AVI does not enhance the flexibility of the manufacturing line because the only use of this
system is for inspection, but the utilization of this system considerably enhances the automation capability
of manufacturing process. The automated vision system can be used for the purpose of measurements,
gauging, integrity checking, and quality control. In the area of measurements and gauging, the gauging of
small gaps, measurements of the object dimension, alignment of the components, and the analysis of
crack formation are common applications. Integrity checking in automotive plants, food industry and
other production lines is performed by using such a vision system.

The medical and pharmacological products can be inspected by the machine vision systems. Using
such an inspection method in the production line has increased the speed and reliability of the
inspections. For example, during the automotive assembly, a vision guided robot recognizes the
orientation of the engine heads and picks and places them correctly on the engine blocks .In another case,
a system examines the fiber optics assembly line.
A PC-based imaging system integrates hardware and software to analyze the captured images for
the possible fiber blemishes, chips, and cracks. As an another example, in aerospace industry, a vision-
based robot using the self-calibrating and self-teaching techniques has been reported that punches rivets
into the airplane metal sheets with the high accuracy .

3.2. Process control


Utilizing a vision system could help a better analysis, control, tracking, and the issuing documents
in different applications. Applying vision system also can help the analysis of the nozzle-plates, and
monitoring production process. It also provides on-line inspection and imaging options for the biomedical,
pharmaceutical, metal finishing, lumber production, and also in the automotive production assembly
lines.

3.3. Parts identification


Parts identification and classification are one of the most important applications of a vision
system. Sorting of the automotive castings, parts, and identifying and unloading of parts from pallets are
important applications. Sorting and grading of the food and other products are another example of such
identification applications.

3.4. Robotic guidance and control


The automation process is based on the different robots, which require guiding systems.
Alignment and adjustment processes also require smart guidance systems. In general, in the automotive
plants guidance of the robotic action is performed by using the vision system as smart sensors for position
determination in the welding or other processes.
4. Objective

The objective is to design a flexible automatic industrial machine vision system for inspection and
assembly that uses a neuro-fuzzy based approach to perform image processing. This type of system is of
particular interest to companies in the automotive sector where time, cost, and reliability are important
factors. Besides , the automotive industry also tends to have more variable production environment than
traditional machine vision inspection systems.

The salient features of the system may be as follows :

1. ANFIS may be used in the recognition module of the vision system for inspection.

2. Segmentation method using ANFIS should decrease the time required to train automated inspection
system and also increase reliability for difficult inspection tasks.

3. Soft computing tools may be applied in the different stages of image processing (image segmentation,
feature extraction, object recognition and image understanding) wherever applicable and the results will
be compared with the traditional image processing techniques.

4. By combining the methods of neural networks and fuzzy the logic, it is possible to create a system
which can learn from a set of training data, but encode that knowledge in a fuzzy structure so that the
results will be more predictable and the system will be more robust to noise.

The developed alogirthm will be used in the online inspection of weld quality of Cu-Cr-Zr alloy
using high speed camera and aids in monitoring of the welding process.
5. References

1. Feldmann K, Slama S. Highly flexible Assembly Scope and Justification. CIRP Annals -
Manufacturing Technology 2001; 50:489-498.

2. Teck LW, Sulaiman M, Shah HNM, Omar R. Implementation of Shape Based Matching Vision
System in Flexible Manufacturing System. Journal of Engineering Science and Technology Review 2010;
3(1):128-135.

3. Kurada S, Bradley C. A review of machine vision sensors for tool condition monitoring.
Computers in Industry 1997; 34(1): 55-72.

4. Wu WY, Wang MJJ, Liu CM. Automated inspection of printed circuit boards through machine
vision. Computers in Industry 1996; 28(2): 103111.

5. Sahoo SK, Choudhury BB. A Robotic Assistance Machine Vision Technique for an Effective
Inspection and Analysis. International Journal of Electrical and Computer Engineering (IJECE) 2015; 5(1):
46-54.

6. Chen H, Liu Y. Robotic assembly automation using robust compliant control. Robotics and
Computer-Integrated Manufacturing 2013; 29: 293300.

7. S. Haykin, Neural Networks, Prentice-Hall, Englewood Cliffs, NJ, 1999.

8. N.K. Bose, P. Liang, Neural Network Fundamentals with Graphs, Algorithms, and Applications,
McGraw-Hill, New York, 1996.

9. J. Chang, G. Han, J.M. Valverde, N.C. Griswold, J.F. Duque-Carrillo, S.E. Cork, Quality
classification system using a unified image processing and fuzzy-neural network methodology, IEEE Trans-
actions on Neural Networks 8 (1997) 964973.

10. T. Sarkodie-Gyan, C.W. Lam, D. Hong, A.W. Campbell, An efficient object recognition scheme
for a prototype component inspection, Mechatronics 7 (1997) 185197.

11. Y.H. Chen, Computer vision for general purpose visual inspection: a fuzzy logic approach,
Optics and Lasers in Engineering 22 (1995) 181192.

12. J.C. Bezdek, J. Keller, R. Krisnapuram, N.R. Pal, Fuzzy Models and Algorithms for Pattern
Recognition and Image Processing, Kluwer, New York, 1999

13. J.S. Jang, ANFIS: adaptive-network-based fuzzy inference systems, IEEE Transactions on
Systems, Man and Cybernetics 23 (1993) 6384

Anda mungkin juga menyukai