Anda di halaman 1dari 8

IJSTE - International Journal of Science Technology & Engineering | Volume 3 | Issue 09 | March 2017

ISSN (online): 2349-784X

Early Detection of Borer Insects Tomatoes by

IoT based on Smart Farming System
Ramkumar. R Kaliappan.S
PG Student Assistant Professor
Department of Computer Engineering Department of Computer Engineering
Kumaraguru College of Technology Coimbatore, India Kumaraguru College of Technology Coimbatore, India

PG Student
Department of Computer Engineering
Kumaraguru College of Technology Coimbatore, India

Every year farmers practice huge losses due to pest plague in crops & this in turn impacts his living. In this paper we discuss a
novel advance to solve this problem by regularly monitoring crops using video dispensation, cloud computing and robotics. The
paper concentrates in methodologies to categorize pests in one of the most admired fruits in the world the tomato. An approaching
into how the thought of the Internet of Things can also be conceptualized in this project has been elaborated.
Keywords: Early Pest Detection, Fruit Plucking, Growth Stages, Internet of Things (IOT), Tomato Detection, Video


For a increasingly developing nation such as India, agriculture is one of the most significant sources of income to the country [1].
It must be noted that agriculture only constitutes almost 22% of the countrys Gross Domestic Product (GDP), which is indeed a
significant number [2]. It is due to this fact, that governments of several developing countries set aside a large part of their economic
budget for the growth, development and introducing systematic and technologically sophisticated methodologies for farming each
year. In a farmers perspective, such initiatives would assist them in producing better surrender and safeguarding their manufacture,
thus improvising their lifestyles as well. However, even though such programmes exist, farmers extend to be plagued by three
major areas of concern insufficient water supply (irrigation), attack of crops by pests & insects and thirdly - failure in properly
storing manufacture which in turn might be attacked by pests and rodents [3].
Over the past few years, a lot of research has been carried out to overcome the aforesaid problems. Usage of cloud seeding
techniques and construction of barrage or dams near villages have reduced the water disaster to some extent [4]. Modern storage
techniques which include utilizing rodent exterminators keep rodents at cove [5]. But when it comes to prevention of vermin or
insects from attacking crops, the usual practice is to spray continual amounts of pesticides. Though efficient this practice comes
with its own set of disadvantages. Extreme use of pesticide can lead to beyond repair damage to the environment. Being at the top
end of the food chain, this would also create dangerous health issues to humans, including birth defects. With respect to farmers,
direct contact to these chemicals often lead to skin diseases and extended use might lead to cancer [6]. A solution to this issue, is
by spraying bug killer on crops only when need required and also by eliminating those crops or fruits already affected by vermin.
This solution though simple, would require a constant monitoring by the farmer which would turn out to be infeasible due to
various reasons such as the lack of the agriculturist at all points of time. Ill health of farmers might also constitute for his/her lack
from the farm.
Therefore, in order to address this major issue, we describe a work of fiction, innovative and a fully automated method to perform
a constant observation of the farm at all points of time, even in the absence of the farmer. The major attentiveness of this paper lies
in identifying fruits of crops in their early immature stages and also to monitor them for any disease or bug infestation. If in case
any disease or bug activity is detected, such fruits could be immediately eliminated in turn preventing spreading of the same to
other healthy crops. Pesticides, insecticides and also bio-control of pests [7] could be utilized for immediate removal. For all
experiments conducted, we utilized the Tomato plant (Solanum lycopersicum), since it is one of the most widely grown crops in
the world [8s]. Tomatoes commonly get affected, in their growing stages (unripe stages), by a pest known as the borer insect -
Helicoverpa armigera [9], which is usually recognized by a black color hole in the fruit through which it enters [10]. In order to
identify the same, a complete video processing application using Java was developed and was deployed using cloud computing
which has been detailed further in this paper. The next section describes few other existing methods using real time video
processing to identify immature tomatoes. Section III of this paper discusses the setup used for our testing and also that deployed
on the cloud. The complete video processing algorithm developed is elaborated in section IV along with the results obtained in
section V. Section VI deals with the future scope of this research with some insights into further extensions.

All rights reserved by 532

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)


As of date, a lot of researchers have discussed new methodologies to notice fruits and vegetables on plants utilizing various
techniques such as video processing and have also explained systems which can fortitude or segregate the same. The first ever
publication to discuss the same, was in the year 1995 [11]. In this paper, the authors introduce a work of fiction idea to identify
tomatoes in plants using several visual sensors and also to fortitude the same using suction cups attached to a robot. Unfortunately,
no further explanation on how to achieve the same has been elaborated. In [12] a complete explanation of automatic tomato
recognition on plants using a normal laser and an infrared laser has been described. The system though accurate proves to be cost
ineffective. [13] - [15] utilize image processing techniques and hi-fi cameras to detect tomatoes but involve difficult and time
consuming algorithms such as principal component analysis and K-Means clustering, which slow down the complete process.
Several other research works exist which deal with the detection of oranges [16], pomegranates [17] and the egg plant [18] but use
techniques not applicable to this project. Several methodologies to identify tomatoes in controlled environments within industries
on conveyor belts also exist [19]-[21], but cannot be utilized since these approaches do not address the problem of intervention of
leaves, stems and other environmental factors. It must also be noted that all the papers mentioned above deal exclusively with
detection of ripe fruits in turn helping in harvesting, whilst our scope of research deceit in the fact that detection has to be done at
the immature stages itself. Also, there is limited amount of discussions on detection of vermin and more particularly the borer
insect. The next section discusses the Setup used for our conducting tests in detail. The setup described can be used in combination
with the robot explained in [22].


The initial setup basically consists of a camera which is mounted on a robotic car. The robotic car is designed such that it can move
around the field, in the space among two rows of crops, with the camera capturing images or obtaining a live video give food to of
the crops. For the primary experiments conducted, we utilized an Intex 5.0 mega pixel web camera with an image resolution of
356x320 [23]. The camera was located at a distance of 1.5 feet absent from the tomato plants. The total frame rate of the same was
30 frames per second. This in turn was connected to a laptop on which the main video processing algorithm runs. Based on the
decisions taken by the algorithm, a robotic arm or a spraying system could be designed to spray the necessary amount of pesticide
on the tomatoes. Though this proved to be a highly speed well-organized solution, the main disadvantage of this setup was the
overall power obsessive. Since the camera, motors of the robot and also the laptop have to be powered using a moveable power
supply, such a solution would prove to be difficult to the farmer, since a constant watch on the power levels of the battery has to
be taken care of and everyday recharges become necessary.
In order to take care of this problem and by doing so, reduce the power consumed by the system, it was predictable to think of
a solution in which the processing the most power keen task of the system [24], could be performed elsewhere. In other words,
create a system which could wirelessly communicate with the camera and the robot and do the necessary processing on a remote
server. Therefore, in order to achieve the above mentioned necessities we utilized concepts of cloud computing as part of our setup.
An illustration of the setup in the playing field using a wireless web camera can be seen in Fig. 1.

Fig. 1: Design of the setup by means of a wireless web camera

According to the National Institute of Standards and Technology (NIST) [25], cloud computing is distinct as a model which is
capable of creating a universal, all over the place available, on-demand network access to a distantly located shareable group of
computer resources that can be utilized with minimal involvement of the end user. Any cloud computing setup utilizes three major
service models which are found in our setup as well. The complete cloud computing explanation for our system can be seen in Fig.
2, comprehensive of the three major service model layers The Software as a service (Saas), the platform as a service (Paas) and
the Infrastructure as a service (Iaas) [26]. The same has been elaborated below.

All rights reserved by 533

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)

Infrastructure as a service (IaaS)

Infrastructure as a service mainly defines the hardware or the processor on which the applications on the cloud would be executed.
For all our experiments an Intel i3 32-bit core processor was utilized.

Platform as a service (Paas)

In order to organize the application created on the hardware mentioned above, there is a need of a software operating system on
which the expansion kit or the necessary libraries for the applications to work can be installed. In this case we utilized the Microsoft
Azure software as the Paas.

Software as a service (Saas)

The software application developed and which is to be deployed on the Paas/Iaas is nothing but the Saas. In this case, the Java
application accountable for detecting the unripe fruits and the borer bug using the wireless feed from the camera constitutes the
Saas in our system.

Fig. 2: Cloud computer based architecture for smart agricultural

The complete algorithm of the application developed has been elaborated in detail in the next section.


As mentioned before, a novel and a highly speed well-organized video processing algorithm was developed using the Java
programming language. Speed effectiveness was necessary since decisions on whether to spray the pesticide or not has to be taken
directly and sent back to the robot from the cloud. The first phase of the algorithm deals with the actual immature tomato detection
on the plant and later phase of the algorithm concentrates on finding the borer bug. The steps followed for the same have been
shown in the flowchart in Fig 3. And the explanation for the same has been described in detail below.

Color Conversion
Each frame of the real time video feed obtained from the system explained in the previous section is first subjected to color change.
It must be noted, that the frames obtained are in Red, Green and Blue color system. Though it is of common idea that green
tomatoes could be directly identified using the green color plane, it is not true. The main reason for the same is the dependence of
the RGB color space on brightness [27]. Hence, if the brightness changes, the range of the RGB values on behalf of the immature
tomato also changes, making it impossible to identify tomatoes under different brightness situation. Therefore, in order to
recompense for this effect, the frames obtained are first converted to a model known as the YCbCr model [28]. The conversions
are protected by equations (1)-(3) as defined by the ITU-R BT.601 [29] and have been given below.
Y' = (0.299 R) + (0.587 G) + (0.114 B) (1)
Cb=0.16*R0.33*G+0.500*B (2)
Cr = 0.5*R - 0.418688*G - 0.081312*B (3)

All rights reserved by 534

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)

Fig. 3: Flowchart depicting the steps involved in our algorithm

Where, Y represents the concentration, Cb represents the chroma blue color component in other words blueness of the image
devoid of brightness and Cr the chroma red part in other words redness devoid of brightness. The color converted frames are
then segmented using the Cb and Cr mechanism alone, to obtain the region of interest. Another important advantage of color
change in this case, is that the amount of memory necessary to store each pixel of the pictures reduces to only 16 bits as different
to 24 bits which was used with the RGB model. This helps to decrease the overall memory necessity by 1/3 rd.

Segmentation of Immature Tomatoes

Segmentation, in image processing, is the task of separating the region of attention from the set. In this case, our region of interest
is exclusively the immature green tomatoes and the challenge deceit in the fact that, this has to be segregated from a background
mainly consisting of stems, leaves and other artifacts false closely in the same range as that of the tomato. Several experiments
were conducted on over 100 images of tomatoes, and it was found that green tomatoes have a separate range of Cb and Cr which
were obtained through histogram analysis. The histograms procured for the same have been shown in Fig. 4. Similar histograms
were obtained for various images taken under changeable environmental conditions.

Fig. 4: Histograms of Cb and Cr of the tomato images with the ranges marked

All rights reserved by 535

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)

Utilizing these values, segmentation was performed in turn making the tomatoes black (0) and the background white (255).
Assuming cr1 and cr2 are the ranges for the Cr components of the object of notice and cb1 and cb2 - the ranges for the Cb
component, then the segmented image S obtained can be represented by (4).

255, otherwise (4)

Noise Removal
As with all segmentation techniques, one of the major issues faced after conclusion of the process, is the incidence of noise. In this
case, a few parts of the stem and leaves are chosen up as part of the tomato and consequently it is necessary to eliminate the same.
In this paper, noise removal was performed using a methodology known as corrosion. A structuring element of 3x3 was applied
on the resultant image S obtained in the previous step. Upon conclusion of this process, most of the noise is obliterated but along
with it comes a major disadvantage corrosion causes even the edges of the tomato to disappear. Therefore, in order to abolish
this effect, we go in for the reverse process dilation. Whilst corrosion is a process of checking whether any surrounding pixel is
black and then making the pixel under consideration black as well (if it is white), dilation on the other hand is quite the opposite
[30]. Hence on application of the dilation algorithm, we keep back the edges and the shape of the tomatoes to some extent. A
structuring element of 13x13 was utilized for this process.

Obtaining back the Tomatoes in Color

After implementation the above mentioned steps, we now require to get back the tomatoes in color, so that further steps can be
processed. In order to perform the same, we utilize the dilated image (D) obtained as a mask and logically AND the same with
the RGB image (C) captured from the camera. By doing so, we now attain only the tomatoes in color and the background remains
black. This can be further represented by (5).
Extracted Image(i,j) = D(i,j) & C(i,j) (5)
Once all these processes complete, phase 2 i.e. the identification of the borer insect on the tomatoes extracted, begins. This has
been explained further in detail.

Identification or segmentation of the borer insect

The final phase of the experiment is to take out the borer insect, if present on the tomato. In order to perform the same, the color
image obtained above (after masking) is now subjected to color change once again. Since borer insects are indicated by black color
holes on the tomato, it is sufficient therefore to convert the color image to an strength component (Y). This is due to the fact that,
once histogram analysis is performed, the pixels belonging to the borer would always recline in the former end of the histogram.
This can be seen in Fig. 5.

Fig. 5: Histogram of the extracted tomato with borer insect

Utilizing (4) and the new threshold values, successful origin of the borer insect was possible. An corrosion operation with a
structuring element of size 3x3 and a dilation of structuring element size 7x7 was performed to retrieve the borer with exactness.
A simple counting algorithm could now be initiated, to count the total number of borer insects affecting the fruit or an area
calculation can also be performed to find the extent of damage. The next section presents the results obtained for the algorithm
explained in this part.

All rights reserved by 536

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)


As mentioned in Section III, all algorithms were designed and developed using the Java programming language in combination
with the Java Development Kit (JDK) 1.6 and Java Media Framework (JMF) 2.1. All programs were deployed as part of the Saas
layer of the cloud computing standard on a 32- bit Windows system. Testing was conducted on over 100 images of tomatoes
(immature) procured from actual farms and also from the internet. Fig. 6 shows an example of an immature tomato picture taken
1.5 feet away from the plant.

Fig. 6: Immature tomato picture in use 1.5 feet away

It can be clearly seen that the top right most tomato has been exaggerated by the borer bug whilst the bottom left few tomatoes
are still undamaged. As explained in the previous part, the first step would be to change the input image into the Cb and Cr
components correspondingly. Fig. 7a and 7b show the same.

Fig. 7: Cb and Cr components of the Tomato picture procured

The image obtained after applying the histogram based segmentation can be seen in Fig. 8.

Fig. 8: Image obtained after histogram based segmentation

All rights reserved by 537

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)

As discussed earlier, there is an excessive amount of noise in the image. The results obtained after the corrosion and dilation
operation can be seen in Fig 9a and Fig. 9b respectively. The regions which contain the tomato can also be seen clearly. Once this
procedure completes, the next step is to perform masking using the mask obtained after dilation.

Fig. 9: Images obtained after erosion and dilation

The resultant image obtained after masking can be seen in Fig. 10.

Fig. 10: Image obtained after masking

It can be clearly seen that the immature tomatoes have been successfully extracted and the borer recognition algorithm can be
performed. Extraction of the borer follows the steps mentioned in the previous part. It can be visualized from Fig. 11, that the borer
bug has been successfully detected. The non-affected tomatoes have been completely disregarded.

Fig. 11: Borer insect successfully extracted from the tomato

Further to this, a speed relationship i.e. a complete comparison of the implementation time of our algorithm with that of the
algorithms explained in [15] & [14] to detect green tomatoes, was performed and tabulated in Table 1.
Table - 1
Comparison of Timings of Three Algorithms

It can be clearly seen from Table I that the algorithm designed and implemented in this paper takes a mere 0.46s in comparison
with the other two well-known algorithms which were used in [14] and [15]. This would in turn assist in a faster judgement of
tomato borer infestation and action can be taken more quickly.

All rights reserved by 538

Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System
(IJSTE/ Volume 3 / Issue 09 / 106)


In this paper a novel methodology to perform smart agricultural and also to detect the borer bug on tomato plants, has been
described. The complete model implemented on a cloud computing framework was also elaborated. It can also be seen that the
algorithm explained is 1.71 times faster than two other popular methods of immature tomato finding. Based on the resultant
obtained from the algorithms, a choice can be taken as to how much amount of bug killer should be sprayed. This prevents
unnecessary spraying of pesticides as well as protection the health of farmers, consumers and also the environment. Future versions
of this implementation could include a unique IP address for the robot, in turn controlling the same via cloud computing or by a
farmer elsewhere. In other words a complete Internet of Things move toward can be utilized. Governments of various countries
can adopt this plan and provide reprieve to hundreds of farmers worldwide.

[1] Huddar, S.R., Gowri, S., Keerthana, K., Vasanthi, S., Rupanagudi, S.R.,"Novel algorithm for segmentation and automatic identification of pests on plants
using image processing," Computing Communication & Networking Technologies (ICCCNT), 2012 Third International Conference on , vol., no., pp.1,5, 26-
28 July 2012
[2] Ministry of Statistics and Programme Implementation, Press Note on Quarterly Estimates of Gross Domestic Product for the Third Quarter (October-
December) of 2013-14, Government of India, February 2014.
[3] The International Rice Research Institute, Suphan Buri, Thailand, 1974 and 1975, in Constraints to High Yields on Asian Rice Farms: An Interim Report,
Philippines, 1977, Ch 7, pp 255.
[4] Mohana A. Krishnan, The promise of cloud-seeding, The Hindu, April 13, 2014. [Online]. Available: the-
promise-of-cloudseeding/article5906186.ece [Accessed Sept.10, 2014].
[5] Sahay, K.M., Singh, K.K., Storage, in Unit Operations of Agricultural Processing, Noida, India: Vikas Publishing House, 2004, Ch 4, pp 164- 172.
[6] World Health Organisation, Preventing Disease through Healthy Environments, Exposure to Highly Hazardous Pesticides: A Major Public Health Concern,
Geneva, 2010.
[7] Singh, A., Parmar, N., Kuhad, R.C, Biological Control of Pests, in Bioaugmentation, Biostimulation, Biocontrol, Springer Science & Business Media,
2011, Ch 10, pp 223-225.
[8] Ghaffari, R.,, "Early detection of diseases in tomato crops: An Electronic Nose and intelligent systems approach," Neural Networks (IJCNN), The 2010
International Joint Conference on, vol., no., pp.1,6, 18-23 July 2010.
[9] Pea, J. E., Biology, Ecology and Management of the South American Tomato Pinworm, Tuta absoluta, in Potential Invasive Pests of Agricultural Crops,
Volume 3 of CABI invasive species series, CABI, 2013, Ch 6, sec 1, pp 98.
[10] Missouri Botanical Garden, Tomato Fruit Problems, Last updated: April 2012.[Online]. Available:
gardening/yourgarden/help-for-the-home-gardener/advice-tips-resources/visualguides/tomato-fruit-problems.aspx. [Accessed Sept. 10, 2014].
[11] Kondo, N., Monta, M., Fujiura, T., Shibano, Y., "Intelligent Robot to Harvest Tomato," Robotics and Automation, 1995. Proceedings., 1995 IEEE
International Conference on , vol.3, no., pp.V19,, 21-27 May 1995
[12] Gotou, K., Fujiura, T., Nishiura, Y., Ikeda, H., Dohi, M., "3-D vision system of tomato production robot," Advanced Intelligent Mechatronics, 2003. AIM
2003. Proceedings. 2003 IEEE/ASME International Conference on , vol.2, no., pp.1210,1215 vol.2, 20-24 July 2003.
[13] Yang, L., Dickinson, J., Wu, Q. M J, Lang, S., "A fruit recognitionmethod for automatic harvesting," Mechatronics and Machine Vision in Practice, 2007.
M2VIP 2007. 14th International Conference on , vol., no., pp.152,157, 4-6 Dec. 2007.
[14] Kundu, S., Hazra, A, Deb, K., Hazra, P., "Dimensionality reduction of morphological features of tomato leaves and fruiting habits," Communications, Devices
and Intelligent Systems (CODIS),2012
[15] Hongpeng Yin, Yi Chai, Yang, Simon X., Mittal, G.S., "Ripe tomato extraction for a harvesting robotic system," Systems, Man and Cybernetics, 2009. SMC
2009. IEEE International Conference on , vol., no., pp.2984,2989, 11-14 Oct. 2009
[16] Cakir, Y., Kirci, M., Gunes, E.O., Ustundag, B.B., "Detection of oranges in outdoor conditions," Agro-Geo-informatics (Agro-Geo-informatics), 2013 Second
International Conference on , vol., no., pp.500,503, 12-16 Aug. 2013
[17] Akin, C., Kirci, M., Gunes, E.O., Cakir, Y., "Detection of the pomegranate fruits on tree using image processing," Agro-Geoinformatics (Agro-
Geoinformatics), 2012 First International Conference on , vol., no., pp.1,4, 2-4 Aug. 2012
[18] Jian Song, "Target identification based on automatic threshold for eggplant picking robot," Information Science and Engineering (ICISE), 2010 2nd
International Conference on , vol., no., pp.1567,1569, 4-6 Dec. 2010
[19] Gejima, Y., Houguo Zhang, Nagata, M., "Judgment on level of maturity for tomato quality using L*a*b* color image processing," Advanced Intelligent
Mechatronics, 2003. AIM 2003. Proceedings. 2003 IEEE/ASME International Conference on , vol.2, no., pp.1355,1359 vol.2, 20-24 July 2003
[20] Guo Feng, Cao Qixin, "Study on color image processing based intelligent fruit sorting system," Intelligent Control and Automation, 2004. WCICA 2004.
Fifth World Congress on , vol.6, no., pp.4802,4805 Vol.6, 15-19 June 2004
[21] D Amato, Juan Pablo, Garcia Bauza, Cristian, Boroni, Gustavo, "Color Based Fruits classification using GPU," Latin America Transactions, IEEE (Revista
IEEE America Latina) , vol.9, no.3, pp.346,350, June 2011
[22] Nezhad, M.AK.B., Massh, J., Komleh, H.E., "Tomato Picking Machine Vision Using with the Open CV's Library," Machine Vision and Image Processing
(MVIP), 2011 7th Iranian , vol., no., pp.1,5, 16-17 Nov. 2011
[23] Intex Technologies, Intex IT-305WC 5MP Webcamera [Online]. Available: [Accessed Sept. 10, 2014].
[24] Azhar, S., Vasanthi, S., Rupanagudi, S.R., "Low Power Optimization of a 3G UMTS Filter," International Conference on Frontiers of Computer Science.
(ICFOCS), 2011, vol., no., pp., August 2011
[25] National Institute of Standards and Technology, The NIST Definition of Cloud Computing, Recommendations of the National Institute of Standards and
Technology, Special Publication 800-145, Gaithersburg, 2011.
[26] Shao, J., Wang, Q., Mei, H., Model Based Monitoring and Controlling for Platform-as-a-Service (PaaS), Cloud Applications and Computing (IJCAC),
International Journal of , vol. 2, issue 1, 2012.
[27] Ravoor, P.C., Ranjani, B.S., Rupanagudi, S.R., "Optimized fingertip blob recognition for image processing based touch-screens," Recent Advances in
Computing and Software Systems (RACSS), 2012 International Conference on , vol., no., pp.104,108, 25-27 April 2012
[28] Dhruva, N., Rupanagudi, S.R., Sachin, S.K., Sthuthi, B., Pavithra, R., Raghavendra, "Novel segmentation algorithm for hand gesture recognition,"
Automation, Computing, Communication, Control and Compressed Sensing (iMac4s), 2013 International Multi-Conference on , vol., no., pp.383,388, 22-23
March 2013 [29] Radiocommunication Sector of ITU, Recommendation ITU-R BT.601- 7, Studio encoding parameters of digital television for standard 4:3
and wide-screen 16:9 aspect ratios, March 2011.
[29] Gonzalez, R.C., Morphological Image Processing, in Digital Image Processing, India: Peasrson Education, 2009, Ch 9, sec 2, pp 652-656

All rights reserved by 539