DETECTION OF PAVEMENT DISTRESS USING LASER TECHNOLOGY

SEMINAR REPORT

Submitted in partial fulfillment of the requirements for the award of M.Tech Degree in Civil Engineering of University of Kerala

Submitted by

BHAGEERATHY K P
M2 Traffic and Transportation Engineering Roll No: 122607

DEPARTMENT OF CIVIL ENGINEERING

COLLEGE OF ENGINEERING
TRIVANDRUM
2013

DEPARTMENT OF CIVIL ENGINEERING

COLLEGE OF ENGINEERING
TRIVANDRUM
2013

CERTIFICATE
This is to certify that this seminar report entitled DETECTION OF PAVEMENT DISTRESS USING LASER TECHNOLOGY is a bonafide record of the work done by Bhageerathy K P under my guidance towards the partial fulfillment of the requirements for the award of M.Tech Degree in Civil Engineering (Traffic and Transportation Engineering) under the University of Kerala during the year 2013.

Guided by

Professor (PG Studies)

Dr. Manju V S Associate Professor Department of Civil Engg. College of Engineering Trivandrum

Dr. M Satyakumar Professor Department of Civil Engg. College of Engineering Trivandrum

ACKNOWLEDGEMENT

I am sincerely indebted to my guide Dr. Manju V. S., Associate Professor, Department of Civil Engineering, College of Engineering Trivandrum, for her valuable guidance and suggestions in preparing this seminar report. I would also like to thank Prof. Jyothis Thomas, Professor and Head, Department of Civil Engineering, Dr. M. Satyakumar, Professor (P.G Studies), Prof. Jayaprakash Jain, Staff Advisor and Prof. Leema Peter, Assistant Professor (Project coordinator), Department of Civil Engineering, for their encouragement and support. I would also like to express my sincere gratitude to all my friends who supported and helped me in completing this report.

BHAGEERATHY K. P.

ABSTRACT

There are several reasons which contribute towards pavement distresses. Detection and repair of distresses on time is very much necessary for preventing the failure of pavements. Usual method of detection of distresses using human observations is extremely tedious and prone to errors. To overcome the limitations of visual evaluation, several attempts have been made to automate the process of detection. One such technology is by using laser scanners. They can be used effectively for getting 3D pavement surface data. Two case studies were dealt with for understanding the application of laser technology in distress detection. The first study deals with the detection of potholes and severity classification using laser technology. The second study evaluates the feasibility of using laser technology to detect cracks. Both studies show that laser technology is very promising for pavement distress detection.

CONTENTS
Page No. 1. INTRODUCTION 1.1 General 1.2 Objectives 2. PAVEMENT DISTRESSES 2.1 General 2.2 Types of pavement distresses 2.2.1 Surface defects 2.2.2 Cracks 2.2.3 Deformation 2.2.4 Disintegration 3. LASER TECHNOLOGY 3.1 General 3.2 Basic concept 3.3 Principle of 3D laser scanners 4. CASE STUDY 1 4.1 General 4.2 Need for the study 4.3 Methodology 4.4 Image processing techniques for detecting potholes using laser pattern 4.4.1 Multi-window median filtering 4.4.2 Tile partitioning 4.4.3 Laser line deformation detection approach 4.5 Pothole severity classification 4.6 Experimental results 13 13 14 15 16 18 1 1 2 2 2 2 2 3 6 8 9 9 9 10 12 12 12 12

4.7 Concluding remarks 5. CASE STUDY 2 5.1 General 5.2 System set-up 5.3 Experimental tests 5.3.1 Controlled test procedure 5.3.2 Field test procedures 5.4 Findings and concluding remarks 6. CONCLUSIONS REFERENCES

19 20 20 20 22 22 25 27 28 29

LIST OF FIGURES
Figure No. Title Page No.

Fig 2.1 Fig 2.2 Fig 2.3 Fig 2.4 Fig 2.5 Fig 2.6 Fig 2.7 Fig 2.8 Fig 2.9 Fig 2.10 Fig 3.1 Fig 3.2 Fig 4.1 Fig 4.2 Fig 4.3 Fig 4.4 Fig 4.5

Fatty surface Streaking Alligator crack Longitudinal crack Edge crack Shrinkage crack Slippage Rutting Corrugation Potholes Illustration of optical triangulation principle Principle of 3D laser scanners Deformed laser pattern on detecting a pothole Four masks used for filtering Image thresholding Template laser line Template matching method for pothole shape estimation

3 3 4 4 5 5 6 6 7 9 10 11 13 13 14 15

16 18

Fig 4.6

Architecture of the neural network

Fig 4.7 Fig 5.1

Results extracted from image database Sensing vehicle integrated at the Georgia Institute of Technology

19

21

Fig 5.2

Laser crack measurement system and projection of laser 21 22

Fig 5.3 Fig 5.4

Visualization of 3D pavement surface data A gap between two solid wood boards to simulate a crack with known width

23 24 24 26

Fig 5.5 Fig 5.6 Fig 5.7

Two lighting conditions Crack segmentation results on simulated cracks Test results on a crack with low intensity contrast

LIST OF TABLES

Table No.

Title

Page No.

Table 4.1 Table 4.2 Table 5.1 Table 5.2

Distress classification guideline Severity level comparison Scores for the controlled tests Scores for the second field test

18 19 25 27

1. INTRODUCTION

1.1 General Quantification of pavement distresses like cracks, potholes etc. is one of the most important tasks in determining optimal strategies of pavement maintenance. These distresses, which are caused by several reasons, curtail the service life of pavements. Once initiated, distress increases in severity and extent, allowing water to ingress the pavement. Over the past years, a significant amount of efforts has been spent on developing methods to objectively evaluate the condition of pavements. For the inspection of the surface distress of highway pavements, the most widely used method to conduct such surveys is based on human observation. This approach is extremely labour-intensive, prone to errors, and poses hazards. To overcome the limitations of the subjective visual evaluation process, several attempts have been made to develop automatic procedures. Most systems use optic images and vision technology to automate the process. However, due to the irregularities of pavement surfaces, there has been a limited success in accurately detecting distresses and classifying distress types. In addition, most systems require complex algorithm with high levels of computing power. With the advancement in sensor technology, an advanced 3D laser system has become available. The 3D laser scanning is one of the exceptionally versatile and efficient technologies for accurately capturing large sets of 3D coordinates. 3D laser scanner uses a technique that employs reflected laser pulses to create accurate digital models of existing objects. For 3D survey, detection of pavement distresses, such as potholes or patches, is possible application where laser scanner technology excels. The advancement of the scanner has invoked many applications such as civil engineering, natural hazard investigations, heritage, landscape design, and tunnel and cave survey, and pipelines. The most popular applications include archeology, as-built surveying, re-modeling of tunnels, bridges, and other civil structures, topographic mapping for base maps, engineering design, and mining fields, and reconstruction of traffic accidents, and urban planning.

1.2 Objectives The objectives of the study are: i. To know about various distresses occurring on pavements. ii. To understand the concept of laser technology. iii. To study about the applications of laser technology in detection of distresses in pavement.

2. PAVEMENT DISTRESSES
2.1 General This chapter describes the various distresses that occur in pavements. 2.2 Types of Pavement Distresses The various types of pavement distresses are grouped as: Surface defects: which include fatty surfaces, smooth surfaces, streaking and hungry surfaces Cracks: under which hair line cracks, alligator cracks, longitudinal cracks, edge cracks, shrinkage cracks, and reflection cracks comes Deformation: which includes grouped slippage, rutting, corrugations, shoving, shallow depressions, and settlements and upheavals Disintegration: covering slipping, loss of aggregates, raveling, potholes and edge breaking 2.2.1 Surface defects These are associated with the surfacing layers and may be due to excessive or deficient quantities of bitumen in these layers. Fatty surface: Fatty surface shown in Fig 2.1 results when the bituminous binder moves upward in the surfacing and collects as a film on the surface. The causes for a fatty surface are excessive binder, loss of cover aggregates in surface dressing, non uniform spreading of cover aggregates, too heavy prime or tack coat and excessive heavy axle load.
2

Fig 2.1 Fatty surface (Source: Google image) Smooth surface: A smooth surface has a very low skid resistance and becomes very slippery when wet. A primary cause for the smooth surface is the polishing of aggregates under traffic. Also excessive binder can result in a smooth surface. Streaking: Streaking is characterized by the appearance of alternate lean and heavy lines of bitumen either in longitudinal or transverse direction. It is shown in Fig 2.2.

Fig 2.2 Streaking (Source: Google image) Hungry surface: Hungry surface is characterized by the loss of aggregates from the surface or the appearance of fine cracks. One of the reasons for hungry surface is the use of less bitumen in the surfacing. Sometimes this condition may also appear due to use of absorptive aggregates in the surfacing. 2.2.2 Cracks Formation of cracks is a common defect in bituminous surfaces. The crack pattern can, in many cases, indicate the cause of the defect. The common types of cracks are:
3

Hair-line crack: These appear as short and fine cracks at close intervals on the surface. These cracks are caused by insufficient bitumen content, excessive filler at the surface and improper compaction. Alligator crack: These appear as interconnected cracks forming a series of small blocks which resemble the skin of an alligator as shown in the Fig 2.3. Main reasons for alligator cracks are excessive deflection of the surface over unstable subgrade, excessive overload by heavy vehicles, inadequate pavement thickness ageing of binder etc.

Fig 2.3 Alligator crack (Source: Google image)

Longitudinal crack: These cracks appear more or less, on a straight line along the road. The reasons may be due to alternate wetting and drying beneath the shoulder owing to poor drainage or due to depressions in the pavement edge. It is shown in Fig 2.4.

Fig 2.4 Longitudinal crack (Source: Google image)

Edge crack: They are found parallel to the outer edge of the pavement as shown in Fig 2.5. They are caused by lack of lateral support from the shoulder, settlement of underlying material inadequate surface drainage, shrinkage, frost heave etc.

Fig 2.5 Edge crack (Source: Google image) Shrinkage crack: These are cracks appearing in the transverse direction, or as interconnected cracks forming a series of large blocks. The pavement itself appears to have no deterioration or deformation, but it is the top surfacing that seems to have become old and cracked. The primary cause for such cracks is the shrinkage of the bituminous layer itself with the age. It is shown in Fig 2.6.

Fig 2.6 Shrinkage crack (Source: Google image) Reflection crack: They are the sympathetic cracks that appear in the bituminous surfacing over joints and cracks in the pavement underneath. The pattern may be longitudinal, transverse, diagonal or block. They occur most frequently on overlays on cement concrete pavements or on cement-soil bases. They may also occur in overlays on flexible pavements where cracks in the old pavement have not been properly repaired.

2.2.3 Deformation Any change in the shape of the pavement from its original shape is a deformation. Slippage: It is the relative movement between the surface layer and the layer beneath. It is characterized by the formation of crescent-shaped cracks as shown in Fig 2.7, that point in the direction of the thrust of the wheels on the pavement surface. It is caused by unusual thrust of wheels in a particular direction, inadequacy of tack coat or prime coat, lack of bond between the surface and the lower course due to excessive deflection etc.

Fig 2.7 Slippage (Source: Google image) Rutting: It is a longitudinal depression or groove in the wheel tracks as shown in Fig 2.8. Ruts are usually of the width of wheel path. If rutting is accompanied by adjacent bulging, it may be the sign of subgrade movement or weak pavement. The causes of rutting are heavy channelized traffic, inadequate compaction of the mix, improper mix design, weak pavement, intrusion of subgrade clay into base layer etc.

Fig 2.8 Rutting (Source: Google image)

Corrugation: It is the formation of fairly regular undulations across the bituminous surface as shown in Fig 2.9. It cause discomfort to motorists. Its causes are lack of stability in the mix, excessive binder, high proportion of fines, round and smooth aggregates, soft binder and faulty laying of surface course.

Fig 2.9 Corrugation (Source: Google image) Shoving: It is a form of plastic movement within the layer resulting in localized bulging of the pavement surface. Shoving occurs characteristically at points where traffic starts and stops such as intersections, bus-stops etc. The first indication of shoving is the formation of slippage cracks which are crescent shaped cracks. Shoving can be caused by lack of stability in the mix, lack of bond between bituminous surface and underlying layer, heavy traffic movement of a start and stop type, use of non-volatile oil on roller wheels etc. Shallow depression: They are localized low areas of limited size, dipping about 25mm or more below the desired profile, where water will normally collect. If not rectified in time, they may lead to further deterioration of the surface and cause discomfort to traffic. They are caused by the settlement of lower pavement layers. Settlement and upheaval: They are characterized by large deformations of the pavement. They are extremely uncomfortable to traffic and cause serious reduction in speed. The causes are inadequate compaction of the fill at locations behind bridge abutments, excessive moisture in the subgrade, inadequate pavement thickness and frost heave conditions.

2.2.4 Disintegration There are some defects which if not rectified immediately, result in the disintegration of the pavement into small, loose fragments. Disintegration, if not arrested in the early stages, may necessitate complete rebuilding of the pavement. Stripping: It is characterized by the separation of bitumen adhering to the surfaces of the aggregate particles in the presence of moisture. This may lead to loss of bond and subsequently to loss of strength and materials from the surface. The reasons for stripping are use of hydrophilic aggregates, inadequate mix composition, continuous contact of water with the coated aggregate, over heating of aggregate or binder, presence of dust or moisture on aggregate when it comes in contact with the bitumen, occurrence of rain or dust storm immediately after construction, opening the road to fast traffic before the binder has set, use of improper grade of bitumen, ageing of bitumen etc. Loss of aggregate: It occurs in surfaces which have been provided with surface dressing. The surface presents a rough appearance, with some portions having aggregates intact and others where aggregates have been lost. The loss of aggregates can occur due to ageing and hardening of binder, stripping of binder from aggregates due to wet weather, wet or dusty aggregate, insufficient binder, aggregate having no affinity to the binder, insufficient rolling before opening to traffic, cold-spraying of bitumen or delaying the spreading of aggregates over sprayed bitumen. Ravelling: Ravelling is generally associated with premixed bituminous layers. It is characterized by the progressive disintegration of the surface due to the failure of the binder to hold the materials together. The raveling process generally starts from the downwards or from edge inward. It usually begins with the blowing off of the fine aggregates leaving behind pock marks on the surface. The reasons for raveling are inadequate compaction, construction during wet or cold weather, use of inferior quality of aggregates, insufficient binder in the mix, ageing of binder, poor compatibility of binder and aggregate, over heating of mix and improper coating of aggregates by the binder.
8

Pot-hole: Pot-holes are bowl-shaped holes of varying sizes as shown in Fig 2.10 in a surface layer or extending into the base course caused by localized disintegration of material. They usually appear after rain. The reasons for formation of potholes are ingress of water into the pavement through the surfacing course, lack of proper bond between the bituminous surfacing and base, insufficient bitumen content, too thin a bituminous surface which is unable to withstand the heavy traffic, too much or too few fines.

Fig 2.10 Potholes (Source: Google image) Edge-breaking: The edge of the bituminous surface gets broken in an irregular way, and if not remedied in time, the surfacing may peel off in large chunks at the edges. The causes of edge breaking are infiltration of water, worn out shoulders resulting in insufficient side support to the pavement, inadequate strength at the edge of the pavement due to inadequate compaction etc.

3. LASER TECHNOLOGY

3.1 General This section describes the laser technology that can be used for collecting 3D pavement profile data for detecting the distresses in pavement. 3.2 Basic Concept Many 3D data acquisition systems are on the basis of the triangulation principle, as shown in Fig 3.1. In such systems, a specific and often fixed pattern of
9

illumination (i.e., structured illumination) is projected onto an object to be measured. The structured light that is projected is a laser line for the system proposed. A digital area scan camera with a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) sensor is placed at a known distance and an oblique angle () with respect to the light projector. The camera takes images of the structured light. Then the deformations of the laser line on the object are analyzed to evaluate the depth (z-axis) for each point with a known horizontal position (x-axis) on the object. The 3D system is usually coupled with an encoder, which enables the system to obtain the y-axis position (i.e., the driving direction). Consequently, a complete three dimensional set of points of the objects surface can be acquired. In addition, such 3D systems can provide both depth and intensity information. The intersection between the emitted structured light and the field of view of the digital camera defines the measurement range of a 3D system.

Fig 3.1 Illustration of optical triangulation principle (Source: Tsai and Li, 2012)

3.3 Principle of 3D Laser Scanners A 3D laser scanner, also known as LiDAR (Light Detection And Ranging), can be considered as an auto-scanning total station which is able to acquire thousands
10

of points in a few seconds. The laser scanner is operating to obtain point coordinates referenced to an internal coordinate system as shown in Fig 3.2. On basis of the mode of platform, LiDAR can be ground-based, air-borne, or space-borne. Usually, the effective distances between the scanner and the objects are in a short-range (< 1 m), mid-range (1-30 m), long-range (30 m-1 km), or super-range for air-borne system (600 m-3000 m). One of the advantages of 3D laser scanner is to measure 3D coordinates of a complicated object in a distance which may hinder further physical contact of the object.

Fig 3.2 Principle of 3D laser scanners (Source: Chang et al., 2011) Ground-based LiDAR is composed of a high pulse rate laser ranger and a directional mirror, thus to accurately measure the range from laser head to the target and then computed along with the mirror angles to obtain the 3D coordinates of the target. Typical 3D scanner system is composed of three parts of components: 1. Laser ranger: including transmitter, receiver, detector, amplifier, timing counter and other electronics. To assure that laser light pulse is transmitted and received in a defined field of view, the light is transmitted and received in an identical path. 2. Optical or mechanical scanning components: for guiding the light in a specified direction, e.g. rotation mirror, plan rotating mirror. 3. Control and data processing components: including computer, and softwares for scaning control, preprocessing and post processing such as data processing and analysis.

11

4. CASE STUDY 1 DETECTION OF POTHOLES USING LASER TECHNOLOGY

4.1 General This case study done by Yu and Salari (2011) deals with the detection of potholes and its severity measurement using laser imaging. 4.2 Need for the Study Over the years, Automated Image Analysis Systems (AIAS) have been developed for pavement surface analysis and management. The cameras used by most of the AIAS are based on Charge Coupled Device (CCD) image sensors where a visible ray is projected. However, the quality of the images captured by the CCD cameras was limited by the inconsistent illumination and shadows caused by sunlight. To enhance the CCD image quality, a high-power artificial lighting system can be used, which requires a complicated lighting system and a significant power source. 4.3 Methodology The proposed laser-based optical system consisted of an active light source that projects a line pattern of laser beams onto the pavement surface, a camera for capturing images, and the image processing algorithms that identify the potholes, as shown in Fig 4.1. After the pavement images were captured, regions corresponding to potholes were represented by a matrix of square tiles and the estimated shape of pothole is determined. Following the pothole detection, a feed-forward neural network is used to determine its severity. The vertical, horizontal distress measures, the total number of distress tiles and the depth index information are calculated providing input to a three-layer feed-forward neural network for pothole severity. To validate the system, actual pavement pictures were taken from pavements both in highway and local roads, and experiments were done.

12

Fig 4.1 Deformed laser pattern on detecting a pothole (Source: Yu and Salari, 2011) 4.4 Image Processing Techniques for Detecting Potholes Using Laser Pattern The main aim of the image processing module is to extract laser color regions in the image. After extracting the laser line from the background, the resulting image is searched for any deformation in the shape of the laser line pattern. The different image processing techniques for detecting potholes using a laser pattern are: 4.4.1 Multi-window median filtering The removal of impulse noise is an important issue in pothole detection. After the video sequences are captured as image frames, the frames are scanned to detect the laser line. The laser line is affected by the superposition of a certain amount of undesired external lighting. A multi-window median filter is applied in the initial step to perform noise reduction in an image. The standard median (MED) filter is a wellknown nonlinear filter that eliminates the noise and performs well in the smooth region of an image. Since the detection of pavement distress involves the detection of liner structures in the pavement image, a multi-stage median filter which uses 4 directional median values as represented by masks in Fig 4.2, is considered.

Fig 4.2 Four masks used for filtering (Source: Yu and Salari, 2011)

13

The multi-stage median filter could be used to reduce the noise while preserving much of the detail in the 2-D image and produces comparable results with the standard median filter. 4.4.2 Tile partitioning Thresholding: Image tiling starts with binarizing the image using a thresholding operation. Thresholding is a widely used technique for image segmentation and feature extraction. In many applications of image processing, the gray levels of pixels belonging to the object are substantially different from the pixels belonging to the background. During the thresholding process, individual pixels in an image are marked as object pixels if their value is greater than some threshold value and as background pixels if lower. In this study, a laser line pixel is given a value of 0 while a background pixel is given a value of 1. Finally, a binary output image is created, as shown in Fig 4.3.

Fig 4.3 Image thresholding (Source: Yu and Salari, 2011) Noise removal: In this step, morphological closing is applied in order to fill small holes, bridge the thin gaps in the binary image, connect nearby laser line pixels without changing the laser line area significantly, and smooth the boundaries. The noise in the binary image is reduced by labeling connected components and counting the number of connected pixels. The operation scans the image and groups them together into components based on pixel connectivity, i.e. all pixels in connected component share similar pixel intensity values and are connected with each other. Based on the number of pixels in a connected component, any connected components less than a predefined value would be considered as noise and removed from the image.

14

Tile partitioning: The method proposed in this study relies on sub-images of pavement rather than pixels. The output image from the previous step is divided into 625 sub-images called tiles. Each tile is 40x40 pixels which covers a 2x2 inch block on the pavement surface. The tile-based method significantly reduces the computational complexity relative to pixel-based computations. As a result, it is less affected by background noise because a few noise pixels alone would not be sufficient for a tile to be classified as a pothole tile. After the binary image is sub-divided into square tiles, each tile is classified as either a laser line tile or non-laser line tile. The decision to classify a tile as a laser line tile is based on the global mean value versus the local mean value of each tile. Any tile that has a mean value lower than the global mean value is considered as a laser line tile and would be labeled 1; otherwise it would be labeled with 0. In this way, a tile-based matrix is generated. 4.4.3 Laser line deformation detection approach The laser line in the pothole area of the image produces a visible contour of a deformed pattern. For example, the projection of a laser line onto a plane area produces a pattern with a different shape than the projection of a laser line onto a ball. The deformation of the laser pattern can reveal the presence of the pothole. In order to detect the deformation of the laser line, a template matching method is used. A predefined laser line template is generated as shown in Fig 4.4. The input frames are then compared with the predefined template frame to detect the deformation.

Fig 4.4 Template laser line (Source: Yu and Salari, 2011)

15

Template matching method: Pothole shape estimation: After the tile partitioning step, each frame is compared with the template frame, tile by tile, for any deformation. The number of tiles in each row that differ between the input frame and the template frame are calculated. If the row that has the maximum deformation of 1s is above the row that has the maximum deformation of 0s, the laser line would be intersecting an obstacle in the scene. Otherwise, the row is determined to be an actual deformed tile row due to a pothole. The row is stored in a new matrix as the first row. This matching process will continue until no further deformation is detected. All rows that qualify for deformation are stored in the new matrix. The output matrix would be an estimated shape of the pothole. The process is shown in Fig 4.5.

Fig 4.5 Template matching method for pothole shape estimation (Source: Yu and Salari, 2011) Depth index: Depth information is defined based on the extent of the deformation that affects rows in each frame. For example, in Fig 4.5 (a), deformation could be detected in 2 rows, so the depth information of this frame would be 2. All depth information is stored frame by frame until no further deformation is detected. The average of this information is computed as the depth index for the detected pothole and stored for further analysis. 4.5 Pothole Severity Classification Distress extracted through the process as explained in the previous section can be classified into different types of potholes with various severity levels (low,
16

moderate, or high). In the study, a three-layer feed-forward neural network is used for distress classification. The distress measure in an image is calculated by accumulating the differences between adjacent histogram values. The neural network distinguishes the distress type by finding the unique pattern of uniformity in these distress values. Histograms are used to measure the statistical information by counting the number of distress tiles (zeros) in each column, row and the whole matrix. The vertical distress measure is determined by accumulating the differences between the numbers of deformed tiles in adjacent columns using equation,

(4.1) Where VD is the vertical distress measure, Hv is the vertical histogram, and Nc is the number of columns, respectively. Similarly, the horizontal distress measure is computed by accumulating the differences between the number of deformation tiles for adjacent image rows using equation,

(4.2) Where HD is the horizontal distress measure, h is the horizontal histogram, and Nr is the number of rows, respectively. If both horizontal and vertical distress measures are having large value, the distress is classified as a pothole. A neural network is used for pothole classification. Four parameters (the vertical distress measure, the horizontal distress measure, the total number of distress tiles and the depth index) are used to provide the inputs to a feedforward neural network. The architecture of the neural network which has a total of 4 input nodes, 8 hidden nodes and 5 output nodes is shown in Fig 4.6. The severity level of the pothole is classified according to the data shown in Table 4.1.

17

Fig 4.6 Architecture of the neural network (Source: Yu and Salari, 2011)

Table 4.1 Distress classification guideline (Source: Yu and Salari, 2011) Distress type Low Moderate High Vertical distress measure >5 >5 >5 Horizontal distress measure >5 >5 >5 Total No. of distress tiles < 40 40-120 > 120 Depth index 1 2 3

4.6 Experimental Results The proposed algorithm was implemented in MATLAB R2008b on a set of over 100 images (10 images for each distress) taken from the road surface. Fig 4.7 shows sample distress classification results extracted from the image database. Fig 4.7 (a) and (b) shows two typical road surface scans of pothole images. Fig 4.7 (d) and (e) shows the pothole image represented by tiles.

18

Fig 4.7 Results extracted from image database (Source: Yu and Salari, 2011) Table 4.2 summarizes the rating results from manual and proposed laser-based approaches. It can be clearly seen that, in all tested samples, the severity level and the crack type detected by the proposed method is in agreement with the level obtained from the manual method. Table 4.2 Severity level comparison (Source: Yu and Salari, 2011) Sample No. 1 2 Distress type Pothole Pothole Severity Level or Crack Type Manual Moderate Moderate Proposed Moderate Moderate

4.7 Concluding Remarks In the study, a laser based pothole detection and classification method using advanced image processing techniques was used. It has been shown that the proposed system allows complete automation with the evaluation of pavement potholes. In comparison with other existing 2-D pavement distress detection and classification methods, the proposed method has a better ability to discriminate the dark areas that are caused by lane marks, oil spills, or shadows. The experimental results indicated that the proposed system provides reliable and accurate results from the tested samples.
19

5. CASE STUDY 2 DETECTION OF ASPHALT PAVEMENT CRACKS USING 3D LASER TECHNOLOGY

5.1 General This study conducted by Tsai and Li (2012) was sponsored by the U.S. Department of Transportation (US DOT) Research Innovative Technology Administration (RITA) program. It deals with the evaluation of feasibility of using emerging 3D laser technology to detect cracks under different lighting and poor intensity contrast conditions. 5.2 System Set up A sensing vehicle as shown in Fig 5.1 was integrated at the Georgia Institute of Technology for collecting 3D pavement surface data. First, the 3D system, composed of two high-performance laser profiling units, was mounted on the vehicle. The field of view of the two units covered a full lanes width. The acquired 3D laser profile was designed to have a 15 clockwise tilt angle to the pavements transverse direction, as shown in Fig 5.2. This was to ensure that 3D transverse profiles can intersect with transverse cracks. Each profiling unit consisted of a 3D laser profiler that uses a high-powered laser line projector, a custom filter, and a camera as the detector. The profiling unit uses a light stripe, which is created by a 7W multiple emitter laser diode and line-generating optics. The light stripe is projected onto an objects surface, and its image is captured by the area scan camera. From the captured image, range measurements are extracted. With a two-unit setup, the Laser Crack Measurement System (LCMS) produces 4,160 3D data points per profile (2080 pixels2 units) covering a 4m pavement width. The resolution in x direction (transverse profile direction) is approximately 1 mm (4 m/4096 points). The accuracy is 0.5 mm in z direction (elevation). The highest resolution in y direction (longitudinal) depends on the distance measurement instrument (DMI) and the accompanying encoder. In the integrated sensing vehicle, an encoder with 1024 pulses per revolution was installed to trigger the acquisition of 3D continuous transverse profiles. The interval between two 3D transverse profiles can achieve 2.3mm by using the encoder. The system can
20

collect transverse profiles at 4.6mm intervals at a speed of 100 km/h. The 3D pavement surface data was then be acquired for detecting cracks. Fig 5.3 shows the visualization of 3D pavement surface data and a closer look at a crack line.

Fig 5.1 Sensing vehicle integrated at the Georgia Institute of Technology (Source: Tsai and Li, 2012)

Fig 5.2 Laser crack measurement system and projection of laser (Source: Tsai and Li, 2012)
21

Fig 5.3 Visualization of 3D pavement surface data (Source: Tsai and Li, 2012)

5.3 Experimental Tests Utilizing the integrated sensing vehicle, experimental tests were conducted to consistently and quantitatively evaluate the feasibility of using 3D laser technology to detect pavement cracks under different lighting conditions and low contrast conditions. Two series of tests were conducted. One was the controlled laboratory test on simulated cracks with known crack widths and depths, and the other was the field test on real roadways. In the controlled tests, the objective was to assess the capability of the 3D laser technology to detect different widths of cracks under different lighting conditions. Four crack widths (1, 2, 3 and 5mm) under two extreme lighting conditions (daytime and nighttime) were tested in the Georgia Institute of Technologys campus laboratory. The crack depth was approximately 19 mm. 5.3.1 Controlled test procedure A controlled gap between two solid wood boards was used to simulate a pavement crack on the road. The width of the gap was measured before and after the test with a caliper, as shown in Fig 5.4. Afterwards the integrated sensing vehicle was driven over the road section by an operator to automatically collect the 3D laser data
22

of both wood boards by using the 3D sensor system. With the 3D laser data, the dynamic optimization was employed to segment the simulated cracks. Meanwhile, the ground truth was manually digitized and extracted from the 3D laser data. The tests were conducted during the daytime and nighttime, as shown in Fig 5.5. Table 5.1 shows the test results. Fig 5.6 shows only part of the controlled laboratory test results. It includes four subsets of figures. Each subset of figures shows the 3D raw data on the left and the crack segmented image produced by using the dynamic optimization algorithm on the right. Fig 5.6(a) and 5.6(b) show the data collected for a 1mm wide crack during the daytime and nighttime, respectively. Fig 5.6(c) and 5.6(d) show the data collected when the simulated crack is 2mm wide. The test results of 3 and 5mm are similar to the 2mm case. It was observed that, the 1mm wide crack was partially captured by the 3D laser technology, and the 2mm wide crack was fully detected. Table 5.1 lists the quantitative scores derived from the linear buffered Hausdorff scoring method for the cracks with different widths under two lighting conditions. For cracks with widths of 1mm, scores are approximately 64. For cracks with widths equal to or greater than 2mm, scores are better, approximately 93. Daytime and nighttime tests resulted in similar scores. The maximum score difference was 0.2. Thus the preliminary controlled laboratory result demonstrated that the 3D laser system is capable of detecting cracks whose widths are equal to or wider than 2mm.

Fig 5.4 A gap between two solid wood boards to simulate a crack with known width (Source: Tsai and Li, 2012)

23

Fig 5.5 Two lighting conditions: (a) daytime (b) nighttime (Source: Tsai and Li, 2012)

Fig 5.6 Crack segmentation results on simulated cracks (Source: Tsai and Li, 2012)

24

Table 5.1 Scores for the controlled tests (Source: Tsai and Li, 2012) Crack width Score 1 mm Daytime Nighttime 63.9 64.1 2 mm 93.6 93.4 3 mm 93.1 93.0 5 mm 93.3 93.1

5.3.2 Field test procedures In addition to the controlled laboratory test, two field tests on actual roadways were conducted. The first field test was to evaluate the potential of the 3D laser system to detect cracks under low intensity contrast conditions. The second field test was to evaluate the capability of the 3D laser system to detect cracks under different lighting conditions, including nighttime, daytime with shadow, and daytime with no shadow. First test: Fig 5.7(a) shows a roadway image with a low intensity contrast between a crack, approximately 1 to 6 mm wide, and pavement background. The low intensity contrast makes the crack difficult to be detected, even with the human eye. However, the data collected using the 3D laser technology from the same area showed a more distinct contrast between the crack and the pavement background. This is illustrated by Fig 5.7(b) and 5.7(d) collected during day and night respectively. Fig 5.7(c) and 5.7(e) shows the corresponding crack segmentation results. The high scores from this first test, namely 98.3 for daytime and 98.0 for nighttime demonstrated the potential of the 3D laser technology for detecting cracks under low intensity contrast conditions.

25

Fig 5.7 Test results on a crack with low intensity contrast (Source: Tsai and Li, 2012)

Second test: The second field test was conducted on State Route (SR) 80 to evaluate the consistency of using the proposed system in detecting cracks under three different lighting conditions: nighttime, daytime with shadows, and daytime no shadows, as shown in Fig 5.8. Eleven test segments, including 10 longitudinal cracks (cracks A to J) and a transverse crack (crack T), were labeled in the field. Examples of the three lighting conditions are shown in Fig 5.9. Fig 5.10 shows the 3D raw data collected under three lighting conditions and corresponding crack segmentation results for the crack J. Visual observation shows that the crack can be clearly captured by the 3D laser system. The scores obtained are listed in Table 5.2. The three scores for each crack were close to each other. The maximum difference among three scores was also tabulated. The average score difference was found to be 1.9. The difference is very small. Therefore, results of field tests demonstrated that the proposed 3D laser system can perform consistently under different lighting conditions in the field.

26

Table 5.2 Scores for the second field test (Source: Tsai and Li, 2012) Score Crack name Nighttime Daytime with shadow 97.4 96.1 96.8 97.2 97.8 98.0 97.7 96.6 96.3 97.6 96.9 Daytime with no shadow 97.2 95.4 97.2 96.9 97.3 97.5 97.5 97.6 97.4 97.7 97.6 Score difference 1.6 0.7 3.6 2.2 1.3 1.5 2.6 2.2 1.1 2.1 1.7 1.9

A B C D E F G H I J T

95.8 95.5 93.6 95.0 96.5 96.5 95.1 95.4 96.3 95.6 95.9

Average score difference

5.4 Findings and Concluding Remarks Both controlled tests and actual road tests have demonstrated that it is feasible to detect cracks under different lighting conditions and low contrast conditions. Controlled tests showed that cracks with widths equal to or greater than 2 mm can be effectively detected from the pavement background, whereas 1 mm wide cracks can be partially detected. The field tests showed that, for three lighting
27

conditions, the average score difference is less than 2%. Thus the experiment shows that the proposed 3D laser technology is very promising for crack detection.

6. CONCLUSIONS

Pavement distresses are contributed by several reasons. If not attended properly, they may lead to progressive failure of the entire pavement. Hence there is a high need for timely detection of pavement distresses. A 2D intensity-based imaging system is the main data acquisition system that has been used for the past two decades. Its intensity based data acquisition method makes it sensitive to lighting effects. In general, the performance of distress detection is severely hampered in the presence of shadows, lighting effects, non-uniform crack widths, and poor intensity contrast between cracks and surrounding pavement surfaces. The shallow or thin cracks are sometimes invisible to the 2D system. Manual inputs are required to adjust the input parameters so that the algorithms can perform reasonably. Therefore, full automation of pavement distress detection has remained a challenge especially for accurate and reliable detection. With the advances in sensor technology, a 3D laserbased pavement surface data acquisition system that can collect high resolution 3D continuous pavement profiles for constructing pavement surfaces has become available. This 3D laser system is different from current 2D intensity-based imaging systems. First, the 3D laser-based system is not sensitive to lighting. Noise, such as oil stains and poor intensity contrast, will not interfere with the segmentation algorithms by using the acquired range data. As long as there is a distinguishable elevation difference between a crack and its surrounding background, the segmentation algorithm is able to capture the crack. In order to understand the potential application of laser technology in distress detection, two case studies were taken. The first study deals with the automatic detection of potholes using laser-based optical system. The second study evaluated the feasibility of using 3D laser technology to detect cracks under different lighting conditions and low contrast conditions. Both studies showed that laser technology has potential applications in timely and fully automated detection of distresses in pavements.
28

REFERENCES

1. Chang, K, T., Chang J, R., Liu J, K. (2005). Detection of pavement distresses using 3D laser scanning technology, ASCE 2005 International Conference on Computing in Civil Engineering, Maxico, Cancun, July 12-17. pp. 1-11. 2. IRC: 82-1982. Code of practice for maintenance of bituminous surfaces of highways. 3. Tsai, Y, C, J., Li, F. (2012). Critical assessment of detecting asphalt pavement cracks under different lighting and low intensity contrast conditions using emerging 3D laser technology. Journal of Transportation Engineering, Vol. 138 (5), pp. 649-656. 4. Wang, C, P, K. (2000). Designs and implementations of automated systems for pavement surface distress survey. Journal of Infrastructure Systems, Vol.6 (1), pp. 24-32. 5. Yu, X., Salari, E. (2011). Pavement pothole detection and severity measurement using laser imaging. IEEE Journal. pp. 1-5.

29