TECHNOLOGY TREND CLASS ASSIGNMENT

“ AI THAT SEES LIKE HUMAN ”

Object Recognition using Camera For Intelligent

Traffic Transportation Device

Lecturer : Dr.Eng. Wahyul Amien, S.T., M.T.

Written by:

No Name NIM
1 Agus Hartanto 30000318410012
2 Akhmad Abdul Hakim 30000318410011
3 Budi Sulistiyo 30000318410025
4 Edy Umargono 30000318410009
5 Kresna Prasmadewa 30000318410005

MAGISTER SISTEM INFORMASI

UNIVERSITAS DIPONEGORO SEMARANG
2018
 TOPIC : (3) AI THAT SEES LIKE HUMAN

Object Recognition using Camera For Intelligent Traffic Transportation Device

_________________________________________________________
Agus Hartanto (30000318410012), Edy Umargono (30000318410009), Akhmad Abdul Hakim
(30000318410011), Budi Sulistiyo(30000318410025), Kresna Prasmadewa (3000031840005)
* MSI – Diponegoro University

ABSTRACT
Currently the development of microprocessor technology is very rapid (as stated in the
law of Moore), this causes many to be affected, the development and speed of computers
becomes increasing fastly, the storage media is getting larger capacity, so that the impact on
software development will be even faster. Programming methods are also growing, so there
are many programming libraries that support artificial intelligence was increasing fastly too.
As a product of artificial intelligence, there are tools that can behave like human intelligence.
This technology have an impact of the development of many aspect of human civilization.
Many devices were created to help, make it easier, and give entertainment to human. In this
article, we give an example of Artificial Intelligent implementation in transportation world.
During an very complexity of transportation problem, an intelligent transportation devices
was needed, to regulate complex traffic, and solve a complexity transportation problem, but
sometime an intelligent transportation devices have a problem with the sensor, that provide
a data to main processor, this will result the device not being able to work optimally.Image
and video analysis namely computer visual analyses are made to recognize and analyze any
object using camera. Real-time vehicle detection, tracking and counting of vehicles is now
very important, to provide a good transportation system. Utilization of video image
processing algorithm which detect, tracks and find the number of vehicles on a road is very
interesting, because this technology can be implemented easily and not need a special
devices. The camera can be used as a sensor to detect an object. The system developer
only focused on the algorithm and programming to produce a software. The output of vehicle
recognition system, can be used as a sensor for many various purposes, this sensor can be
embedded to another device, including intelligent traffic light, electronic traffic sign, and
electronic instruction for roads user, which can be used to complement the weaknesses of
intelligent transportation devices.

INTRODUCTION artificial intelligence software.

The development of technology in the
21st century is very rapid, especially
developments in the field of computer
technology, many intelligent devices are
created, all of this can not be separated from
technological developments from artificial
intelligence.
Artificial intelligence is actually the result
of the development and use of the scientific
and algorithmic methods, which are
combined and developed into a smart
algorithm that can be implemented into an
Many devices was created that can
replace the role of humans in the fields of
industry, commerce, education, medical,
military, sports and the entertainment world,
all of which have intelligence like humans.
Many industrial machines are connected
with sensors, which can do work
automatically to replace the work previously
done by humans. The automated marketing
system has also been widely applied in the
internet world, this system can analyze the
habits of internet users, then record
contacts, e-mails and so on and then be
given a product offer in accordance with the
tendency of one's behavior. In the world of
artificial intelligence education is also widely
implemented, especially for computer-based
training/learning systems, training
participants can take lessons and practice
using a computer, where the computer will
behave like an intelligent instructor, who
guides participants to achieve progress from
their knowledge.
Computer-Based Training (CBT) involves
the use of a personal or networked computer
for the delivery and access of training
programs. CBT can be synchronous and
asynchronous, as well as online, web-based,
mobile, and distance learning. CBT is
particularly useful when training learners on
a specific computer application, but can also
be built to train learners on general
knowledge or skills. The greatest
disadvantage of CBT is that it is expensive
to develop and deliver, especially for smaller
groups of students. CBT can be more cost
efficient when designed to train a larger
number of students.
Fig 1a : SCAT Shooting trainer
In the world of sports, many artificial
intelligence devices are created, which are
combined with sensor technology for the
purpose of conducting scoring and training,
where athletes will be guided and monitored
automatically by the program, and then the
program will provide advice to achieve the
development of abilities from The athlete, for
example, Computer chess is a game of
computer architecture encompassing
hardware and software capable of playing
chess autonomously without human
guidance. Computer chess acts as solo
entertainment (allowing players to practice
and to better themselves when no
sufficiently strong human opponents are
available), as aids to chess analysis, for
computer chess competitions, and as
research to provide insights into human
cognition. Current chess engine are able to
defeat even the strongest human players
under normal conditions. Whether
computation could ever solve chess remains
an open question (figure 1b.).
Fig 1b: Chess Robot
Another example of implementation of
Artificial Intelligent in sport, Russia made
tool called the SCAT shooting trainer, where
the shooting athlete will be trained and given
input automatically about what he has done
wrong, and what he has to correct based on
the analysis of his movements when
shooting.
In the world of entertainment too many
devices were created, from virtual friends in
the form of software that can chat using
natural language, as well as robots that are
seem like humans, who can accompany us
and be invited to talk naturally, this
phenomena was occurred in some
developed countries, that robot was became
a friend in daily life, that robots can behave
like humans who can be trusted, loyal and
ready to accept our heart sadness, as is
currently being a trend in Japan lately (like in
fig 1.b.).
 Advances in artificial intelligence (AI), deep-
learning, and robotics are enabling new
military capabilities that will have a disruptive
impact on military strategies. The effects of
these capabilities will be felt across the
spectrum of military requirements - from
intelligence, surveillance, and
reconnaissance to offense/defense balances
and even on to nuclear weapons systems
themselves.
Fig 1b: Robot seem like human

In Medical world is a broad field where

innovation plays a crucial role in sustaining
health. Areas like biotechnology,
pharmaceuticals, information technology, the
development of medical devices and
equipment, that have an artificial intelligence
aquity and more have all made significant
contributions to improving the health of
people all around the world. From “small”
innovations like adhesive bandages and
ankle braces, to larger, more complex
technologies like MRI machines, artificial
organs, and robotic prosthetic limbs, Fig. 1d. : AI in military weapon
technology has undoubtedly made an
incredible impact on medicine. Artificial intelligence technology is now
In the healthcare industry, the making its way into manufacturing, and the
dependence on medical technology cannot machine-learning technology and pattern-
be overstated, and as a result of the recognition software at its core could hold
development of these brilliant innovations, the key to transforming factories of the near
healthcare practitioners can continue to find future.
ways to improve their practice – from better
diagnosis, surgical procedures, and
improved patient care.

Fig 1.e: AI in industry

The results of artificial intelligence

Fig 1c. : AI devices for sugery development research, have now been sold
by many companies in a smart product,
which has been widely implemented in daily
life, ranging from artificial intelligence game
software such as chess games, smart
mobile-based software, until to device that
have a human like capabilities, one of the
artificial intelligences that greatly contributes
to the development of technology is video or
image based artificial intelligence.
The development of image/video
processing technology using computer
vision is currently very developed. Many
programming library is provide facilities that
implement algorithms, useful for developing
computer vision based applications
Video processing systems use high
performance computer and application
oriented circuits in the current technology. It
is because of the amount of data to be
processed in large and processing this data
in real time is tedious task.
In this article we will discuss about an
algorthm and scientific method that used for
create an video and images processing,
object recognize,etc that the goal is to
produce an intelligent camera sensor, and
we also discuss one of the well-known
libraries in programming languages, which
are widely used for processing video and
images, namely OpenCV.

BACKGROUND
At present many intelligent traffic equipment have been implemented in developed
countries, which are useful for regulating traffic and transportation on the roads and
highways, one of them that very important is traffic lights.
Traffic lights are lights lamp that control the flow of traffic that is installed at a crossroads,
these lights that indicate when the vehicle must walk and stop alternately from various
directions. The calculation of conventional traffic lights does not give the correct green time
lag, the solid track gets a brief green light and the quiet lane gets the old green light. If
reviewed, the use of conventional traffic lights becomes less effective because the road
users must wait for the green light, especially with the density of drivers who spill the lane
and waste of time to get to the destination. Therefore, the application of intelligent
transportation technology using the development of a combined smart traffic system
algorithm with the scientific and algorithm method in which the algorithm reduces and
regulates congestion.
However, currently intelligent traffic light mostly uses wire-based sensors that are planted
on the road, this wire only detects vehicles passing above it, this is certainly still a lot of
weaknesses of this tool relating to input data from sensors whose numbers are inaccurate,
the use of camera-based sensors, the data that becomes the input of intelligent traffic light
will be more accurate. Another used of camera-based sensors can also be used on other
equipment to replace their still conventional sensors, to produce better and more accurate
inputs.

ALGORITHM AND TECHNOLOGY background is extracted from the images.

BEHIND
For the traffic management, vehicles
detection is the critical step. Computer
Vision based techniques are more suitable
because these systems do not disturb traffic
while installation and they are easy to
modify. Image from video sequence are
taken to detect moving vehicles, so that
The extracted background is used in
subsequent analysis to detect and classify
moving vehicles as light vehicles, heavy
vehicles and motorcycle (Nilesh,Ravindra ,
2013)
Image processing technology, which is
implemented by the Opencv library, uses
many algorithms, including the K-NN (K-
Nearest Neighbors) algorithm, MOG
(Mixture of Gaussian), CNN (Convolutional
Neural Network), OpenCV is still developing
and does not rule out possibilities in the
future, there will be many algorithms and
other scientific methods to be implemented
in OpenCV.
OpenCV
Open Source Computer Vision Library
(OpenCV for short) distributed under a BSD
style license which allows for royalty free
commercial or research use with no
requirement that the user’s code be free or
open. OpenCV is supported under Windows
and Linux, but the code is well behaved and
has ported to many other operating systems.
OpenCV contains an optimized collection of
C libraries spanning a wide range of
computer vision algorithms, including motion
segmentation and pose recognition, multi-
projector display system, object and face
recognition, and 3D reconstruction, etc. The
broad functional areas supported by
OpenCV include: Basic structures and array
manipulations, Image processing and
analysis, Object structural analysis, Motion
analysis and object tracking, Object and
face recognition, Camera calibration and 3D
reconstruction, Stereo/3D tracking and
statistically boosted classifiers, user
interface and video acquisition support.
( Qingcang Yu a , Harry H. Cheng a, *,
Wayne W. Cheng b , Xiaodong Zhou b, Ch
OpenCV for interactive open architecture
computer vision ).
K-NN (K-Nearest Neighbors)
k-Nearest Neighbors algorithm (k-NN)
is a non-parametric method used for
classification and regression. This algorithm
aims to find the nearest neighbor in an entity
group,
in k-NN, data is projected into a large
dimension space, where each dimension
represents a feature of the data. This space
is divided into sections based on the
classification of learning data. A point in this
space is marked as class c if class c is the
most common classification found on the
nearest neighbor. Near or near neighbors
are usually calculated based on Euclidean
distance.
Matematically the input consists of the k
closest training examples in the feature
space. The output depends on whether k-
NN is used for classification or regression:
● In k-NN classification, the output is a
class membership. An object is
classified by a majority vote of its
neighbors, with the object being
assigned to the class most common
among its k nearest neighbors (k is a
positive integer, typically small). If k =
1, then the object is simply assigned
to the class of that single nearest
neighbor.
● In k-NN regression, the output is the
property value for the object. This
value is the average of the values of
its k nearest neighbors.
k-NN is a type of instance-based
learning, or lazy learning, where the function
is only approximated locally and all
computation is deferred until classification.
The k-NN algorithm is among the simplest of
all machine learning algorithms. Both for
classification and regression, a useful
technique can be used to assign weight to
the contributions of the neighbors, so that
the nearer neighbors contribute more to the
average than the more distant ones. For
example, a common weighting scheme
consists in giving each neighbor a weight of
1/d, where d is the distance to the
neighbors.
MOG (Mixture of Gaussian)
Mixture of Gaussian algorithm is used to
modeling object and background. After the
background is established, the algorithm
begins to detect foreground objects using
connected components algorithm and
calculate its property, e.g., its position,
motion direction, speed, area, contour, and
color histogram. That information are used
to tracking objects in later frames. According
the posterior information gained by tracking,
the MoG background modeling algorithm
can perfect the background to detect
accurate and integrated foreground objects.
The block diagram of the proposed algorithm
is shown in Fig.2a.
With this algorithm, we can extract a
video into background and foreground, so
the targeted object can be recognize and
detected (Xun Wang, Jie Sun, Hao-Yu Peng
, Foreground Object Detecting Algorithm
based on Mixture of Gaussian and Kalman
Filter in Video Surveillance).

CNN (Convolutional Neural Network)

Fig 2a
Mixture of Gaussian algorithm is desribe as
here : If each pixel resulted from a particular
surface under particular lighting, a single
Gaussian would be sufficient to model the
pixel value. If lighting changed over time, a
single, adaptive Gaussian per pixel would be
sufficient. In practice, multiple surfaces often
appear in the view frustum of a particular
pixel and the lighting condition change.
Thus, adaptive Gaussians are important. We
consider the values of a pixel over time, at
any time t, what is known about a particular
pixel is its history:
{ X 1 ,..., X t } = { I ( x 0 , y 0 , i ) :1 ≤ i ≤ t }
Where I is the frame sequence? Some pixel
samples are shown by the (R, G) scatter
plots in Figure 2b.
Convolutional Neural Networks (CNN)
approaches are commonly used with
architectures having multiple layers that are
trained. There are two stages to train the
network: a forward stage and a backward
stage. Firstly, the main goal of the forward
stage is to represent the input image with
the current parameters (weights and bias) in
each layer. The first layers find low-level
features for instance edges, lines and
corners and the other layers find mid-level
and high-level features for instance
structures, objects, and shapes. Then the
prediction output is used to compute the loss
cost to the ground truth labels. Second,
based on the loss cost, the backward stage
computes the gradient of each parameter
with chain rules. All the parameters are
updated based on the gradients and are
prepared for the next forward computation.
After a number of iterations of the forward
and backward stages, the network learning
can be done. (Boukaye Boubacar Traore 1,
2 , 1 Bernard Kamsu-Foguem, 2 Fana
Tangara, Deep convolution neural network
for image recognition)

Figure 2b.

INNOVATION
Innovation and development of artificial intelligence technology, increasing rapidly from
time to time, until now camera-based artificial intelligence has been developed into a device
that functions as a sensor. By using a camera, the object will be recognized and counted, so
this will contribute to the development of other intelligent devices.

RESULT
This paper is written with a little research, that implement OpenCV library using Python
programming language, which became a small prototype for detecting cars using cameras.

Figure 3 : Software capture Figure 4: Hardware Prototipe

This program can detect and count a number of car in the road. The output of this
program can be connected to another smart devices (example: intelligent traffic lights),
which functions as a sensor.

IMPLEMENTATION
The system is using OpenCV image development kits and experimental results are
demonstrated from real-time video taken from single USB Camera. We tested this system on
a laptop powered by an Intel i7 (3 GHZ) CPU and 8GB RAM, and Linux Ubuntu version
18.04 LTS (figure 4).

Figure 5: Example of Implementation

The example of implementation using camera as a sensor, is Intelligent Traffic light, as

can be seen in figure 5. The output of the camera sensor, will be the input for the main
processor in the intelligent traffic system, the data from the sensor will be analyzed based on
certain configurations, which will determine the duration time of the red and green lights, so
that traffic of vehicle will be more optimal, which will minimize the occurrence of traffic
congestion
CONCLUSION
The development of artificial intelligence provides many benefits for human civilization
when used appropriately and wisely. The selection of the right, effective and efficient
technology is needed in implementing artificial intelligence in aspects of human life.
The use of camera-based sensors and video processing has many advantages. This
sensor is also easy to develop and combine with another devices that have artificial
intelligence functions, also in terms of installation, it does not need a lot of devices that must
be installed in certain places, just only use a surveillance camera, which in addition to being
a CCTV recorder, also functions as a sensor that is connected to other devices.
However, the weakness of using a camera as a sensor requires high resources
hardware, because video image processing requires complex mathematical calculations of
images received. More higher the pixel of the camera, and the more objects that will be
recognized, we need more hardware that has higher specifications.

REFERENCE
Boukaye Boubacar Traore 1, 2 , 1 Bernard Kamsu-Foguem, 2 Fana Tangara, Deep
convolution neural network for image recognition.
Nilesh J. Uke Sinhga and Ravindra C,Moving Vehicle Detection for Measuring Traffic Count
Using OpenCV, Journal of Automation and Control Engineering Vol. 1, No. 4,
December 2013.
doi: 10.12720/joace.1.4.349-352
Qingcang Yu a , Harry H. Cheng a, *, Wayne W. Cheng b , Xiaodong Zhou b, Ch
OpenCV for interactive open architecture computer vision ).
Xun Wang, Jie Sun, Hao-Yu Peng , Foreground Object Detecting Algorithm based on
Mixture of Gaussian and Kalman Filter in Video Surveillance.
YunWei, QingTian, JianhuaGuo, WeiHuang, JindeCao, Multi-vehicle detection
algorithm through combining Harr and HOG features.