The Built & Human Environment Review, Volume 1, 2008

Review of Photographic/Imaging Applications in Construction

Yahaya Makarfi Ibrahim 1 and Ammar Peter Kaka 2 1 Makarfi@gmail.com Department of Quantity Surveying, Ahmadu Bello University, Zaria, Nigeria 2 A.P.Kaka@hw.ac.uk School of the Built Environment, Herriot Watt University, Edinburgh, UK

Abstract
Since the time of the Gilbereths, photographic analysis has been increasingly employed to address various site related issues. The usefulness of photographic information and the various methods of analysing them have been widely reported in the literature. So far, however, there is no comprehensive review that collates and critically evaluates the various applications of photographic images and the various techniques for interpreting them. This paper therefore reviews existing literature to present the state-of-the-art in the various applications of images in construction management. Both manual methods based on time-lapse photography and automated techniques based on photogrammetry and computer vision are reviewed. Further research is recommended on the optimum frame rate at which construction site images should be captured. There is also the need to establish the reliability and extent to which automated techniques can be employed to interpret construction image data.

Keywords: construction images, progress monitoring/ review

Introduction
The use of photographic images in the monitoring and evaluation of construction operations is not new. As far back as the 1960s, Fondahl (1960) presented a valuable analysis of photography and its use in the analysis of construction operations. His was by no means the pioneering study. Studies that date as far back as the time of the Gilbreths have applied photography in the analysis of construction operations. Since then, photography has found general acceptance and has been used to address various issues in construction management. Construction companies are increasingly employing photographs to garner information on site related activities. These have been shown to be invaluable in identifying and solving various construction management problems (Abudayyeh 1997). Typical applications include; progress monitoring, dispute resolution, accident investigation, education, quality assurance, etc. Thus, a plethora of studies have been published in the area of images and their applications in the construction management field. In spite of the many publications, there is not a comprehensive review known to the authors on the applications of images in construction management.

Furthermore, advancement in technology and improved computational power has transformed the ways photographic information is captured, analysed, and communicated. Earlier, image data was mainly analysed visually (manual approach). More recently, however, image processing and computer vision techniques are increasingly being employed to analyse and interpret image data (automated approach). This has a direct impact on the various applications of image data. It is therefore important to collate, through a comprehensive review,

99

The Built & Human Environment Review, Volume 1, 2008

the latest developments in research on the various applications of, and techniques used in analysing construction image data. This will identify research gaps and propose new directions for further research.

The objective of this paper is to evaluate both the manual and automated approaches to interpreting images with a view to highlighting their practical validity in analysing construction operations. The paper presents the stateof-the-art developments in the applications of image data in construction management. It examines these applications, and then provides a critical assessment of both the manual and automated techniques of interpreting image data. First, the paper discusses some of the most common applications of image data in construction. Both the manual and automated techniques of interpreting image data are then examined. Finally, the research gaps are identified and directions for further research are proposed.

Role of Photographic Information in Construction Management

Everett et al. (1998) and Ibrahim et al. (2008) highlighted the various applications of photographic/ image data in the management of construction operations. A typical application is in dispute resolution. When disputes arise between the members of the project team, it is essential to resolve them in a timely fashion as prolonging the resolution process will only serve to impact negatively on the time performance of the project. Visual data in the form of photographs provide an accurate representation of the job site situation. Hence, where there is the need to refer back to the records of operations, photographs provide a useful basis for analysing and settling disputes. Visual archives of construction operations provide useful information that could be employed in training workers. This is particularly applicable where time-lapse movies are developed from image archives. Other applications of photographic data are in the investigation of accidents. Photographic records can provide a useful basis for investigating the cause of an accident. This is helpful to the management in formulating better safety measures to prevent future occurrence. Photographic information can also be employed by companies as a marketing tool. Companies can more easily communicate innovative solutions relating to both product and process through images. This has the potential to have more impact than the use of textual means alone.

Manual Methods of Interpreting Construction Image Data

The majority of the applications of construction image data relied on humans to interpret the data through observation. Various such applications have been reported in the literature. For example, Eldin and Egger (1990) demonstrated the benefits of using camera video recorders (a fairly new technology then), and their potential to improve site operations and labour productivity on construction sites. Although the video records provided accurate records of site operations, their study entailed analysing these by reviewing the tapes, first in a fastforward mode to identify a portion of interest, and then at normal speed. Whilst useful in capturing records of construction operations accurately, this approach is simply impractical where long records of up to two or more years of construction operations are involved.

A similar approach for tracking progress on a construction site was reported by Saad and Hancher (1998). They developed a multimedia system called Project Navigator. The system is developed for tracking the progress of a construction project as well as for project documentation. It is an integration of audio, video, graphics and text files. The video files, which were captured using a camcorder were first digitised, and then edited to enhance their quality. Again, the shortcoming here was the enormous human effort required in analysing the image sequence especially when the duration is long.

100

The Built & Human Environment Review, Volume 1, 2008

The use of video sequences for remote access to construction sites had also been reported. Miah et al. (1998) described the use of a mobile video camera in a construction application setting. Their study highlighted the significance of wearable computers in improving communication flows from a construction site to a remote location. The system uses wireless transmitters to send digital images of the construction sites to a remote location. Apart from enhancing communication, this system can be employed to convey progress information to stakeholders who do not have regular access to the site. Although such a system provides a reliable means of documenting and communicating as-built information, it still requires human effort to visualise and interpret the images.

In a similar effort, Nuntasunti and Bernold (2002) presented a prototype website (site-web-site) designed to aid the planning and control of construction projects by providing continuous visual access to the site. Their system consists of a notebook computer, internet access, motion detector, and a video camera. Like the study of Miah et al, the objective is to enhance communication. However, it also facilitates real-time recording and review of project progress and can be used as a surveillance tool if connected to a motion detector. However, a distinction to Miah et als system is that the site-web-site incorporates the concept of time-lapse so that images are captured and stored at some frame rate. Even so, the system still relies on human effort to visualise and retrieve information from the images.

Shih et al. (2006) developed a panorama image database management system (PIDMS) for the management of construction information. The system integrates images with virtual reality technology to provide a monitoring platform for construction activities. The authors argued that in addition to monitoring site operations, the system could be employed as an educational tool since it combines as-built visual information into the construction management database. Like the studies of Nuntasunti and Bernold, the basic concept of time-lapse is used in recording video sequences. Leung et al. (2008) extended these applications by employing long range wireless technology and network cameras to develop a collaborative platform through which project members can communicate among themselves. Again, the key concept of time-lapse is employed, and the means of interpreting these images is still manual and not automated.

Essentially, the concept of time-lapse is employed to address the problem associated with the length of time required to view movies of a construction project. Construction operations are recorded by taking still images at selected time intervals rather than at a real time rate. This means recording images at a frame rate lower than the standard 30fps (frames per second) established by the National Standard Television Committee. Once recorded in this way, the sequence can then be viewed at the standard frame rate during playback, thereby reducing viewing time. A movie so recorded is said to be in time-lapse mode. In this way, one can watch a ten-hour movie recorded at 1fps in just 20 minutes. Abeid and Arditi (2002) gave a useful description of the fundamentals of time-lapse digital photography.

Everett et al. (1998) extended the use of time-lapse photography. Rather than concentrating on worker productivity, they emphasised the use of time-lapse in the monitoring of entire construction operations. They concluded that this approach can be of great benefit in documenting actual project progress and the recordings can be used in the resolution of claims and disputes, for education, public relations, fund raising, and it provides managers with remote access to project progress. The use of time-lapse has been widely employed in monitoring construction projects successfully (Abeid and Arditi 2002, Abeid and Arditi 2003, and Abeid et al. 2003)

101

The Built & Human Environment Review, Volume 1, 2008

However, there are issues that need to be addressed in the use of time-lapse videos. The first relates to the fact that lowering the frame rate during recording has an implication for the amount of detail that can be observed from the time-lapse video. This is particularly so because some amount of detail is always not recorded within the interval between any two consecutive frames. The frame rate therefore has an effect on the level of accuracy in interpreting time-lapse videos. This situation raises a question regarding the optimum frame rate required for the observation of various construction operations.

Studies have come up with different frame rates for the observation of construction operations. Everett et al. (1998) alluded that an interval frame rate of 1fps to 1fp5s is most common for the observation of construction activities. However, the basis for this is not clear. Abeid and Arditi (2002) recommended recording the operation at 60fpm (frames per minute) and playing back time-lapse movies at lower frame rate depending on the need. Their study recognises that the observation of different operations requires different amount of detail and hence, different frame rates, but it fails to come up with the best frame rate required for the accurate observation of each operation. Kang and Choi (2005) studied the error in observation of time-lapse movies using different frame rates and concluded that a time-lapse photo sequence of less than 60-second (1fpm) interval may be used for monitoring construction operations and evaluating productivity with a 70% level of confidence. Although the conclusion suggests that the frame rate of 1fpm may be used to monitor construction operations as well as evaluate productivity, the study examined only worker productivity.

Although time-lapse videos addressed important issues relating to the amount of time required to review a movie, the basic method in the interpretation of the acquired images is based on human observation of the images. The advantages of this approach are clear. However, human observation is prone to error, and is subjective. The way forward is to automate the process of interpreting the images so that unbiased progress information can be automatically extracted from images. The amount of automation that can be achieved relates directly to developments in the fields of photogrammetry, image analysis, and computer vision.

Image Processing Techniques Applied To Construction Management

A number of automated techniques have been employed in the interpretation of construction image data. The first is photogrammetry. Jauregui et al. (2003) noted that photogrammetric surveying is a technique used to generate three-dimensional measurements from two-dimensional photographs taken of an object. The idea of Architectural photogrammetry was first conceived by the Prussian Architect Albrecht Meydenbauer in 1858 (Albertz 2001). Since then, the use of photogrammetry in 3D reconstruction, and measurements of 3D objects has been applied in the field of construction. Recent applications extend to the measurement of bridge deflection (Jauregui et al. 2003), tracking of excavation activities (Trupp et al. 2004) and the measurement of flat elements of buildings under construction (Ordonez et al. 2008). Earlier, Knight and Kaka (1998) had proposed the application of photogrammetry to building modelling and costing and in lighting calculations (Hadwan et al. 2000).

A more recent application of photogrammetry in construction progress monitoring was reported in AbdMajid et al. (2004); Memon et al. (2005); Memon et al. (2006). In this, the authors proposed a model which they called Digitalizing Construction Monitoring (DCM). The DCM is an interactive system that integrates 3D CAD drawings with digital images. The authors used photogrammetric techniques to extract 3D model from digital images, which is then compared with 3D CAD models to extract progress information.

102

The Built & Human Environment Review, Volume 1, 2008

A similar application of photogrammetry in monitoring construction progress was proposed by Bayrak and Kaka (2004); Bayrak and Kaka (2005). Here, the authors proposed the use of a library that contains the full list of elements that make up 3D model of the building. Once construction begins, captured digital images of the actual state of construction are transferred to a model operator. Based on the image contents, the model operator generates 3D model of the current state of the building by utilising the pre-designed library objects. Once this is done, progress measurement based on earned value analysis (EVA) and calculation of interim payments become easy to conduct since information on each object in the library is readily available.

While these systems provide a useful means of enhancing progress measurement on the construction site, the fact remains they still require a great deal of human interaction. The desire is to reduce, as much as possible, the amount of human interaction. This will reduce subjectivity and consequently enhance the quality of the progress data. This highlights the advantages of a fully automated approach based on computer vision (Lukins et al. 2007). It is worth noting however, that although emphasis is given on separate applications of photogrammetry and computer vision in these sections, a hybrid application of both techniques has been reported by Alves et al. (2003); Alves and Bartolo (2006). In these studies, the authors combined photogrammetry and computer vision with biologically based algorithms to develop a system (BioCAD) for the rapid regeneration of 3D computer models from images of existing buildings.

The second topic is image processing/ computer vision. The majority of applications that extract information from visual data use methods from the areas of digital image and video processing, computer vision, contentbased image and video retrieval, and statistics (Brilakis and Soibelman 2005). These techniques are based on the notion that an image is a collection of pixels, filled with a digitally defined colour. Thus, the image can be represented as a three-dimensional matrix (x, y, colour), where the first two dimensions are the coordinates of the pixel and the last defines the colour of that pixel (Abeid and Arditi 2002).

Brilakis et al. (2005) noted that the term image and video processing is used to refer to tasks that try to enhance, restore, reconstruct, filter, compress, modify, or extract useful information from an image. Computer vision, on the other hand is concerned with measurement, recognition, classification, indexing, motion analysis, and much more (Trucco and Kaka 2004). However, as noted by Trucco and Verri (1998), most computer vision algorithms require some form of preliminary image processing. Hence, there is a significant overlap between the two. These techniques have been applied to address various issues in construction management. The following sections discuss these applications.

Productivity Measurement
There are a few related studies on the use of image analysis and/ or computer vision techniques to improve productivity. Everett (1993) developed a video system to improve productivity and safety of crane operations. The system consists of a video camera mounted on the boom of the crane. The camera captures and transmits real-time images of the load and craftsmen which might otherwise be invisible to the operator. Rather than relying on hand signals, the operator can now view the load and hence better coordinate the directions given by ground crew. Experimental results on the testing of the system showed a 16-21% increase in productivity.

103

The Built & Human Environment Review, Volume 1, 2008

A valuable application of image analysis and computer vision techniques in the measurement of equipment productivity was presented by Zou and Kim (2007). They described an image analysis-based methodology to automatically quantify the idle time of hydraulic excavators by tracing its movement across a sequence of consecutive images taken at a frame rate of 1 image every 10 seconds. The first task in this methodology was to extract the excavator from the image. After the extraction, the excavator was then traced across the sequence of images. The final step was to determine whether or not the excavator was actually moving. The methodology was tested on 3 hour long image sequence and the results showed high accuracy.

Quality Assurance
A valuable application of automated computer vision/ image analysis techniques is in the area of quality assurance. This is due to the realisation that traditional quality assessment techniques are mostly based on subjective criteria which are often an obstacle to effective management (Chang and AbdelRazig 1999). A first application is geared towards enhancing the performance of robots in carrying out quality assurance tasks.

Paterson et al. (1997) showed the potential that computer vision has in improving the exterior inspection of large buildings. Their work combines robotics and computer vision techniques to identify and record exterior defects on tower blocks. The authors needed to address two problems in this work. First they had to develop vision algorithms that identify cracks from images taken by a camera mounted on the robot. Second, they needed a means of documenting the position of the robot, which was based on analysis of images of the robot taken by a second camera. The results of testing the algorithms revealed the need for further research in dealing with the problem of clutter (shadows, reflections, changing lightings, etc.) in order to enhance recognition.

Another effort geared towards enhancing the performance of robots in carrying out quality assurance is reported by Navon (2000). This study developed and incorporated a computer vision algorithm that performs real-time quality assurance function, to a floor tiling robot. The robot was able to detect damaged/ defective tile, and was able to place tiles on a straight line with equal distance between adjacent tiles. It is worth noting though, that this study was lab based and the problems of occlusions which come in the form of dust, shadows and changing lighting conditions must be addressed if the system is to be implemented on a real construction site. Notice that the object of analysis here (the tiles) are of relatively small scale in terms of physical size. Imaging techniques have also been employed in the quality assessment of large structure, especially bridges.

Rust detection on Bridge Coatings

There is quite a plethora of studies on automated quality assessment of bridges. For example, Chang and AbdelRazig (1999); Chang and AbdelRazig (2000), and AbdelRazig and Chang (2000) argued that the traditional technique for bridge inspection is subjective, tedious and hazardous to the operator. They proposed a more objective approach based on image pattern recognition and neural networks techniques. The system is essentially a decision support for the assessment of steel bridge coating based on an analysis of visual image data. These are clearly valuable applications. However, Lee and Chang (2005) noted that environmental issues relating to non-uniform lighting and noise on painting surfaces are some of the limitations associated with these applications.

104

The Built & Human Environment Review, Volume 1, 2008

Lee et al. (2006) agreed that the percentage of rust on steel bridge coating surfaces can be reliably computed using image based processing techniques. However, they noted that past studies have only focused on the identification of the extent or degree of rust on the surfaces. They argued that in order to improve efficiency, an algorithm that first identifies the existence of rust in a given image is required to first confirm the presence of rust in an image before the image is further processed to determine the percentage of rust. Notice that the basic premise of these studies is in the detection of rust on bridge coatings.

Crack Detection
Other related studies focused on the identification of cracks on bridges. Tung et al. (2002) developed a mobile imaging system for automating bridge crack inspection. The system is composed of two cameras and an algorithm designed to detect and compute crack positions from images taken by the cameras. In a similar effort, Abdel-Qader et al. (2003) and Abdel-Qader et al. (2005) applied various image processing algorithms on a sample of 50 concrete bridges. The focus of their study was on a comparison of the effectiveness of these algorithms in crack detection. However, their study did highlight the potential application of computer vision techniques in construction management.

Lee et al. (2006) extended this application of image analysis to incorporate robotics. Their system is comprised of a remote controlled robot with cameras and an image processing algorithm. The robot facilitates the acquisition of images of various parts of the bridge, including the underside of the deck. This is useful as it eliminates risks associated with taking images under potentially risky conditions. Similarly, Hutchinson and Chen (2006) presented a method for identifying cracks on concrete structures. Their method is based on statistical image analysis. The promise of their approach was illustrated by applying it to a sample of images of concrete structures with cracks.

Similar to the work of Lee et al. (2006), Yu et al. (2007) incorporated an image processing algorithm to an inspection robot. Although, unlike the work of Lee et al. (2006) where the approach is designed to identify cracks on concrete bridges, here, the focus is on the identification of cracks in tunnels. The results showed a measurement error of 10% or less.

Notice that the studies reviewed above on crack identification focused on the identification of cracks alone. Chen et al. (2006) extended their application to incorporate the measurement of the sizes of the cracks in addition to identifying them. Their system is capable of identifying crack width and analysing the relationship between expansion of the concrete and the crack width. Results of their experiments showed that differences between the manual methods and their automated system are less than 0.05mm. Applications of image processing techniques in the identification of defects in geological features have also been reported. For example, Leu and Chang (2005) reported on a digital image-based approach for tunnel excavation faces. They applied image processing techniques on images of excavation faces to identify faults and joints on excavation surfaces.

Closely related to the above studies is the work Zhu and Brilakis (2008). Here, however, the authors focused on the identification of air pockets for architectural concrete surfaces. They employed image processing techniques to identify the presence of air pockets and their properties in concrete surfaces. These properties include their number, sizes and the area they occupy on the surface. Results of their experiments showed about 86.7% level of precision.

105

The Built & Human Environment Review, Volume 1, 2008

Indexing and Retrieval of Construction Images

Over the years, imaging has become a popular means of project documentation. More and more information is being captured and stored in the form of digital photographs. These provide accurate progress records and asbuilt documentation of the project. It is common practice to refer to these archives to provide evidence in the case of dispute. However, because of the long durations of construction projects, these image data rapidly grows into thousands of images depending on the rate at which they are captured. This trend of ever increasing digital imaging information in the construction industry is due to the affordability, high resolution, ease of use, and industry acceptance of digital cameras (Brilakis and Soibelman 2005).

Searching for a specific image then becomes extremely difficult and time consuming. An approach to ease this task is based on the application of imaging techniques to retrieve images based on their contents. Reported findings from research at the university of Michigan, USA, demonstrate the feasibility of employing image analysis techniques in the search and retrieval of images from image databases (Brilakis and Soibelman 2005; Brilakis and Soibelman 2005; Brilakis et al. 2005; Brilakis and Soibelman 2006; Brilakis et al. 2006). The method most commonly adopted utilises content-based image retrieval concepts to match known material samples with material clusters within the image content. An evaluation of this method revealed that it has an average effectiveness of 86% (Brilakis et al. 2005). This demonstrates its effectiveness as an image retrieval application.

Notice that the units of analysis in the above content-based retrieval techniques are material clusters contained in an image. The implication is that the retrieval of a particular image, based on a particular material in the image will produce images that have that material content, irrespective of the type of structure to which the material applies. In order to enhance the recognition capabilities of these methods, Brilakis and Soibelman (2006) developed an automated content-based shape recognition model. The model is based on clustering techniques related to material and object segmentation. The enhancement is based on the ability of the model to detect the boundary of each detected material in the image. The model then examines the linearity (length/width ratio) and orientation (horizontal/vertical ratio) to evaluate the objects linearity.

In an effort to further enhance the retrieval capability of the earlier models, Brilakis and Soibelman (2008) developed a novel technique that is capable of identifying both linear (i.e. beams, columns) and non-linear (i.e. walls, slabs) structures within the image content of a construction site image. This approach combines both material and shape criteria in the identification process. The model was evaluated based on two measures: precision (defined as the ratio of number of relevant images to number of retrieved images) and recall (defined as a ratio of number of relevant retrieved images to the total number of relevant images). The results showed that out of a total number of 103 images, steel shapes were identified at 91% precision and 98% recall, and concrete shapes were identified at 82% precision and 91% recall. This methodology is robust and it greatly enhances the capability of previous approaches since it is now possible to further differentiate identified materials in an image into linear and non-linear entities. Although the focus of these studies is to address the problem of image retrieval, the techniques used and the results obtained are clear steps towards recognising construction objects in an image.

106

The Built & Human Environment Review, Volume 1, 2008

Recognising the potential of the imaging techniques used in developing image retrieval systems, Brilakis (2006) extended their application beyond that of image retrieval to the automatic retrieval of on-site project information. This approach combines the material and shape recognition models discussed earlier, with a model that crossreferences identified objects with 3D model design. The objective is to reduce the cost and effort required by inspectors in finding the right design and specification information required to inspect a particular segment of the work. However, as noted by the author, this approach has the potential to be employed as an automated progress monitoring model since it allows for the comparison of as-built information with 3D design data.

Other Applications There are many valuable applications of image processing techniques in construction. Alharthy and Koshak (2007) reported on an approach for detecting and extracting tents using aerial images. Their approach which is based on feature extraction is used to count the number of tents and calculate the area covered by the tents. Another application is in the identification of partially exposed metal objects. Umetani et al. (2007) developed a methodology for the identification of the position and orientation of partially exposed metal objects. Quinones et al. (2008) reported on a methodology for the semi-automatic tracking of excavation activities. They applied their algorithms on a real case construction site and concluded that the results were reliable. Other applications of vision-based techniques such as those related to automated people counting (Schofield et al. 1997), geometry analysis of aggregate particles (Chandan et al. 2004), and pavement distress classification (Obaidat and Alkheder 2006) have also been reported. Although less relevant to this study, they demonstrate the potential for computer vision techniques in construction.

A more relevant application of image processing techniques is in the tracking of construction related entities. A few studies have been reported in this area. For example, Brilakis (2008) reported on a valuable application for tracking project related entities such as materials and equipment. Cordova and Brilakis (2008) later elaborated on the tracking of personnel. Both studies emphasised the unobtrusive advantage of vision based tracking and associated cost savings since, unlike other tracking technologies, entities do not need to be tagged.

Progress Monitoring
Efforts geared towards addressing the issue of automatic progress assessment through computer vision have been reported in the literature. These efforts are discussed later. First though, the key challenges associated with the task are analysed. The first issue is that of identifying the best way to capture the images. In general, there are two possible routes for capturing progress images of the site. One possibility is to have fixed cameras directed at the construction site. Any number of cameras could be used from as many locations as necessary to provide adequate coverage. The disadvantages here lie with the inflexibility to adapt to changes in the structure. This is particularly so since portions of the building will become obscured as the structure evolves. There is therefore the need for substantial forethought and planning in deciding where to place the cameras and supporting network infrastructure for complete coverage. However, such an approach offers the possibility of constantly on-demand images, taken from a known location (Lukins et al. 2007).

The other possibility is to have regular surveys conducted on foot. While better quality images could be captured from any given location, this can lead to problems in reconciling the location from which the image was captured. Although, following a strict protocol and utilizing further geo-referencing technologies such as GPS could alleviate this. Even so, capturing images through regular foot surveys will to an extent, serve to undermine the very essence of automation. In addition, although a combination of both approaches could be used, the fixed107

The Built & Human Environment Review, Volume 1, 2008

camera approach seems more plausible as far as automation is concerned, since it lends itself to less human intervention. The issue of coverage is certainly the biggest concern with using a fixed-camera approach. It is simply not feasible to confirm for any given moment absolutely every component. Particularly as the construction progresses and components that were once visible can disappear from the scene. This could be addressed by applying additional knowledge of the building to infer that components are probably complete (for example, if 3rd floor components are seen, then the 2nd floor must be complete). However, it must be acknowledged that this approach will not be totally reliable, since the only way to truly gain confidence that a component is finished is to visually verify it. The ideal solution must combine multiple sources of image and additional information to increase the overall reliability (Ibrahim et al. 2007).

Given a sequence of captured images and the architectural model of a building structure, The task switches to that of reconciling or matching the 3D model of the building with the captured images in order to identify potential deviations (Lukins et al. 2007). However, it should be noted that images of construction sites are rife with heavy clutter, uncontrolled and unforeseeable lighting variations, dynamic target appearance and frequent occlusions. It is the degree to which these can be controlled that will determine the level of success in automating the task of interpreting the image data.

Abeid and Arditi (2002) described a methodology to automatically recognise structural components in a digital picture of a construction site. Their work was a first step towards incorporating an image-processing engine to PHOTO-NET a system designed by the authors (Abeid and Arditi 2002) that monitors construction progress by linking schedules with time-lapse videos of the construction site. Their work was based on an edge-detection algorithm, an image processing operation used to identify the boundary of an object in an image. The developed system is capable of detecting components in an image based on its colour and position.

Wu and Kim (2004) developed and presented the outcome of an algorithm that can detect a structural component in an image taken of a construction site. The objective was to facilitate automatic project control. Their approach entails segmenting objects of interest from the background and other clutter in an image based on edge detection. The authors claimed their algorithm has produced promising results, opening up opportunities for automatic progress control within the construction industry. However, it is worth noting that their work considered only reinforced concrete columns.

Trucco and Kaka (2004) proposed an algorithm for recognising objects and structures in unconstrained outdoor images of construction sites. Their algorithms are useful in determining the location of a structure on construction sites, and for non-textual indexing and retrieval of images from large image databases of construction projects. The authors employed an iconic image matching algorithm for detection based on the Hausdorff distance between training samples of the component of interest and a novel construction scene. This was then able to successfully highlight the presence and locations of those components visible in the scene, whilst accommodating for some variations in scale, lighting and occlusions. However, while detection and localising components is a useful first step, it does not provide any information as to the fundamental changes that occur to the component, nor the region around it. Tracking and recognizing such changes as key events in a sequence of photographs during construction is the key to automatic progress assessment.

108

The Built & Human Environment Review, Volume 1, 2008

Although previous works have applied computer vision techniques to address various construction management issues, Trucco and Kaka were the first to propose a mechanism that integrates computer vision with various project management functions. Their framework (see Figure 1) captures the essential elements in integrating computer vision with project management functions. However, it only depicts and describes high-level interactions and falls short of describing the details which are necessary for the development of a prototype.

Figure 1: Integrating Computer Vision with Management Functions (Source: Trucco and Kaka (2004))

In a similar effort, Podbreznik and Rebolj (2005); Podbreznik and Rebolj (2006); Podbreznik and Rebolj (2007) emphasised the time-consuming nature of manual methods of monitoring construction progress and proposed a method for automatic comparison of on-site building images with the 4D model of a building. Their framework is based on the reconstruction of 3D geometry models from 2D images of the construction site, which is then compared with the product model component of a 4D model. Again, their framework is not detailed enough to allow for the development of a prototype. In addition, unlike Trucco and Kakas framework, which integrates computer vision-based monitoring with project planning, cost and schedule control, and interim payment computations, Podbreznik and Reboljs framework (see Figure 2) only emphasised the progress measurement dimension.

109

The Built & Human Environment Review, Volume 1, 2008

Figure 2: Basic Concept of Automatic Activity Tracking (Source: Podbreznik and Rebolj (2006))

The technique used by Podbreznik and Rebolj (2007) is based on two principles: The first is the recognition of building elements from site images. The second compares recognised building elements with elements contained in the 3D geometric representation of building at a given time. In this way, any observed differences can be reported as variances from the plan, thus providing useful information as to the state of progress.

In a similar approach, Golparvar-Fard and Feniosky (2007) and Golparvar-Fard et al. (2007) presented a methodology for progress monitoring by comparing site photographs with as-planned model of the building. The first step in their proposed approach entails matching the view of the as-planned 3D model with that of the camera from which photographs are taken. Then, using material-based image retrieval algorithms, the photographs are analysed to establish progress status. The next step is to colour-code all elements as either completed, under construction or yet to be constructed in the as-planned photographs. The rest of the task then involves superimposing the colour-coded elements on the site photographs. In this way, any deviations between the as-planned 3D model can easily be visualised.

A more robust progress monitoring system based on computer vision techniques is reported in Lukins and Trucco (2007) and Ibrahim et al. (2008). This is based on two approaches. The first is based on the notion that progress monitoring is based on change occurring through a sequence of images. This approach relates to the vast research area of change detection, which deals with finding significant changes within images. In essence, their approach to tackling the problem first entails aligning a 3D model of the building component to a sequence of images. The objective here is to generate a set of components region around which interesting changes will be anticipated. The next step is to detect change within the component regions, over time. The final step seeks to verify/ confirm that a detected change in a component region is in deed due to the presence of an erected component. Verification is done using a classifier trained to recognise different components. The authors used a test case and found that the system detected 16 of the 17 columns precisely when they were erected. Although the study was applied to concrete columns only, it does hold a promising potential to improve the accuracy of monitoring the erection of structural components. Their study differs from the ones discussed earlier In addition to identifying components, it also seeks to establish the precise times at which components were first visible in the scene. This is clearly a valuable way of monitoring progress since knowing the precise times at which components were erected will pave the way for further analysis of the status of the work.

110

The Built & Human Environment Review, Volume 1, 2008

An important realisation though is that while progress does relate to change, it can mean the appearance or disappearance of a component. The key issue addressed by the authors was that associated with the appearance of objects/ components. While this is a significant step towards detecting changes, thereby inferring some measure of progress, the study failed to address the issue of progress resulting from the disappearance of components in the scene. For, example, progress achieved by the removal of spoil or form work are not addressed in that study.

The other technique is based on the AdaBoost algorithm (Viola and Jones 2001), a technique for object recognition in cluttered scenes which is based on training a set of classifiers on a collection of images containing a component of interest. In their study Lukins and Trucco (2007) trained their classifiers on a set of 100 images of concrete columns and were able to detect quite a large number of columns in a given image as shown in Figure 3. Notice that the identification of columns is not entirely reliable as some scaffolds (at the top of the image) are also identified as concrete columns.

Figure 3Error! No text of specified style in document.: Example Results for Column Detections Employing an Adaboost Classifier (Source: Lukins and Trucco 2007)

Discussion and Conclusion

Research on the applications of construction image data shows a wide spectrum of applications. Typical applications include productivity analysis, accident and claims investigations, education, remote monitoring of construction operations etc. These applications require capturing and playing back video sequences of construction operations. However, interpreting image data of lengthy construction operations based on human observation requires an equal amount of time as the time taken to do the recording. This is a significant challenge since construction operations typically run to months or even years. In an effort to address the

111

The Built & Human Environment Review, Volume 1, 2008

problem, researchers resorted to the use of time-lapse movies. However, the optimum frame rate by which construction operations should be recorded had not been unequivocally reported.

Reported findings suggest that automated methods based on image processing and computer vision techniques have the potential to be employed in tracking the progress of construction. However, the majority of these studies focused on identifying structural members which are relatively easy to verify visually. This is a positive development. The concern though, is that progress measurement is a lot more complicated and goes far beyond tracking structural members. It includes activities related to preparatory works which may not be captured by cameras, but represent a significant measure of progress. Furthermore, while the argument that the essence of full automation may be defeated by using mobile cameras seems plausible, using fixed cameras on the other hand may only provide sufficient coverage of the building envelop but not the significant amount of work carried out within it. This gives rise to the following questions: If photographic data is useful for dispute resolution, accident investigation, e.t.c., which all require different levels of detail, what is the optimum frame rate for capturing the data? Is it possible to perform those functions effectively from a time-lapse movie recorded at a single frame-rate? To what extent can image analysis and computer vision techniques be employed to monitor the progress of construction works?

The work presented in this paper is based on a review of literature. There is, therefore, the need to investigate issues related to practical applications of photographic/ image data in the management of construction operations. Ethical, moral and legal issues surrounding the practical applications of image data need to be appraised. This will provide a framework that defines the various practical challenges associated with image data and analysis. In this way, researchers can better channel their efforts so that the output of their work has more practical significance.

Acknowledgement The authors would like to thank Tim Lukins for useful discussions on the subjects of image processing and computer vision techniques. This research is funded by the UK EPSRC IMRC Outreach grant EP/C535200/1

References
Abdel-Qader, I., O. Abudayyeh, et al. (2003). "Analysis of Edge-Detection Techniques For crack Identification in Bridges." Journal of Computing in Civil Engineering 17(4): 255-263. Abdel-Qader, I., S. Pashaie-Rad, et al. (2005). A Local PCA Algorithm for Inspection of Concrete Bridges. Proceedings of the 2005 International Conference on Computing in Civil Engineering, Cancun, Mexico. AbdelRazig, Y. A. and L.-M. Chang (2000). "Intelligent Model for Constructed Facilities Surface Assessment." Journal of Construction Engineering and management 126(6): 422-432. AbdMajid, M. Z., Z. A. Memon, et al. (2004). Conceptual Digital Monitoring Model For Evaluating the Progress of Work. Proceedings Of The 4Th Conference Of Construction Applications Of Virtual Reality Lisbon, Portugal. Abeid, J., E. Allouche, et al. (2003). "PHOTO-NET II: a computer-based monitoring system applied to project management." Automation in Construction 12: 603-616.

112

The Built & Human Environment Review, Volume 1, 2008

Abeid, J. and D. Arditi (2002). "Linking Time-Lapse digital Photography and Dynamic scheduling of construction operations." Journal of Computing in Civil Engineering 16(4): 269-279. Abeid, J. and D. Arditi (2002). "Time-Lapse Digital Photography Applied to project Management." Journal of Construction Engineering and Management 128(6): 530-535. Abeid, J. and D. Arditi (2002). "Using Colors to Detect Structural Components in Digital Pictures." ComputerAided Civil and Infrastructure Engineering 17: 61-67. Abeid, J. and D. Arditi (2003). "Photo-net: an integrated system for controlling construction progress." Engineering, Construction and Architectural Management 10(3): 162-171. Abudayyeh, O. (1997). "A Multimedia Construction Delay Management System." Microcomputers in Civil Engineering 12: 183-192. Albertz, J. (2001). Albrecht Meydenbauer Pioneer of Photgrammetric Documentation of the Cultural Heritage. Proceedings 18th International Symposium CIPA 2001, Potsdam, Germany. Alharthy, A. and N. Koshak (2007). "Automatic Extraction of Tents During Hajj from Airbone Images to Support land Use Optimisation." Automation in Construction 16: 107-111. Alves, N. M. and P. J. Bartolo (2006). "Integrated Computational Tools for Virtual and Physical Automatic Construction." Automation in Construction 15: 257-271. Alves, N. M. F., P. J. S. Bartolo, et al. (2003). Biologically Based CAD Tool. Proceedings of the RICS Foundation Construction and Building Research Conference, University of Wolverhamton, UK. Bayrak, T. and A. Kaka (2004). Evaluation of Digital photogrammetry and 3D CAD modelling applications in construction management. 20th Annual ARCOM conference, HWU, Edinburgh, UK. Bayrak, T. and A. Kaka (2005). CAD Based, Semi-automatic Data Acquisition Method for Construction Management 5th International Conference on Construction Applications of Virtual Reality (CONVR), Durham, UK. Brilakis, I. K. (2006). Automated On-Site Retrieval of Project Information. 13th EG-ICE Workshop on Intelligent Computing in Engineering and Architecture, Monte Verita, Ascona, Switzerland. Brilakis, I. K. (2008). Vision-Based Tracking of Project Related Entities. Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, Tennessee. Brilakis, I. K. and L. Soibelman (2005). Comparison of Manual and User-Guided Methodologies for the Classification and Retrieval of Construction Site Images. Construction Research congress, Sandiego, California. Brilakis, I. K. and L. Soibelman (2005). "Content-Based Search Engines For Construction Image Databases." Automation in Construction 14: 537-550. Brilakis, I. K. and L. Soibelman (2005). Identification of Materials from Construction Site Images Using Content-Based Image Retrieval Techniques. Proceedings of the 2005 ASCE International Conference on Computing in Civil Engineering, Cancun, Mexico, ASCE. Brilakis, I. K. and L. Soibelman (2006). "Multimodal Image Retrieval From Construction Databases and ModelBased Systems." Journal of Construction Engineering and Management 132(7): 777-785. Brilakis, I. K. and L. Soibelman (2006). Shape Recognition of Linear Construction Entities From Construction Site Images. Joint International Conference on Computing and Decision Making in Civil and Building Engineering, Montreal, Canada.

113

The Built & Human Environment Review, Volume 1, 2008

Brilakis, I. K. and L. Soibelman (2008). "Shape-Based Retrieval of Construction Site Photographs." Journal of Computing in Civil Engineering 22(1). Brilakis, I. K., L. Soibelman, et al. (2005). "Material-Based Construction Site Image Retrieval." Journal of Computing in Civil Engineering 19(4): 341-355. Brilakis, I. K., L. Soibelman, et al. (2006). "Construction Site Image Retrieval Based on Material Cluster Recognition." Advanced Engineering Informatics 20: 443-452. Chandan, C., K. Sivakumar, et al. (2004). "Application of Imaging Techniques To Geometry Analysis of Aggregate Particles." Journal of Computing in Civil Engineering 18(1): 75-82. Chang, L.-M. and Y. A. AbdelRazig (1999). Using Images Pattern Recognition and Neural Networks for Coating Quality Assessment. W78: 1999 - Information Technology in Construction, Vancouver, Canada. Chang, L.-M. and Y. A. AbdelRazig (2000). Bridge Painting Defects Recognition Using Sampling Plans and Image Processing Techniques. Proceedings of the 2000 International Conference on Construction Information Technology, Taking the construction industry in the 21st century, Reykjavik, Iceland. Chen, L.-C., Y.-C. Shao, et al. (2006). "Measuring System For Cracks in Cincrete Using Multitemporal Images." Journal of Surveying Engineering 132(2): 77-82. Cordova, F. and I. K. Brilakis (2008). Automated 2D Tracking of Personnel in Construction Sites. Proceedings of the 10th International Conference on Application of Advanced Technologies in Transportation, Athens, Greece. Eldin, N. N. and S. Egger (1990). "Productivity Improvement Tool: CAMCODERS." Journal of Construction Engineering and management 116(1): 100-111. Everett, J. G. (1993). "CRANIUM: Device For Improving Crane Productivity and Safety." Journal of Construction Engineering and Management 119(1): 23-39. Everett, J. G., H. Halkali, et al. (1998). "Time-Lapse Video Applications For Construction Project Management." Journal of Construction Engineering and Management 124(3): 204-209. Fondahl, J. W. (1960). "Photgraphic Analysis for Construction Operations." Journal of the Construction Division 86(2): 9-25. Golparvar-Fard, M. and P.-M. Feniosky (2007). Application of Visualisation Techniques for Construction Progress Monitoring. Proceeding of the 2007 ASCE International Workshop on Computing in Civil Engineering, Pittsburgh, Pennsylvania, USA. Golparvar-Fard, M., A. Sridharan, et al. (2007). Visual Representation of Construction Progress Metrics on Time-Lapse Photographs. Proceedings of Construction Management and Economics Conference (CME25), Reading, UK. Hadwan, M. K., A. P. Kaka, et al. (2000). "Application of Photogrammetry in Lighting Calculations for Obstructed Interiors." Lighting Research Technology 32(1): 13-17. Hutchinson, T. C. and Z. Chen (2006). "Improved Image Analysis for Evaluating Concrete Damage." Journal of Computing in Civil Engineering 20(3): 210-216. Ibrahim, Y. M., A. P. Kaka, et al. (2008). "As-built Documentation of Construction Sequence by Integrating Virtual Reality with Time-lapse Movies." Architectural Engineering and Design Management 4: 73-84.

114

The Built & Human Environment Review, Volume 1, 2008

Ibrahim, Y. M., T. C. Lukins, et al. (2007). A conceptual progress measurement framework for automatic workpackaging and assessment with computer vision. Proceedings of the 24th W78 conference, & 14th EG-ICE workshop, & 5th ITC@EDU workshop, Maribor, Slovenia. Ibrahim, Y. M., T. C. Lukins, et al. (2008). "Towards Automated Progress Assessment of Workpackage Components in Construction Projects Using Computer Vision." Advanced Engineering Informatics In Press. Jauregui, D., K. R. White, et al. (2003). "Noncontact Photogrammetric Measurement of Vertical Bridge Deflection." Journal of Bridge Engineering 8(4): 212-222. Kang, J. and J.-W. Choi (2005). Observation Error of Time-Lapsed Photos in Construction Operation Monitoring. Proceedings of the ASCE International Conference on Computing in Civil Engineering, Cancun, Mexico. Knight, M. W. and A. Kaka (1998). Application of Computer Based Photogrammetric Techniques to Building Modelling and Costing. Proc ECPPM'98- 2nd European Conf On Product and Process Modelling in The Building Industry, Watford, Building Research Establishment ltd. Lee, B., W. Lee, et al. (2006). Development of Remote Control System For Bridge Inspection Using Robot and Digital Image Processing Technology. Joint International Conference on Computing and Decision Making in Civil and Building Engineering, Montreal, Canada. Lee, S. and L.-M. Chang (2005). Digital Image Processing Methods for Assessing bridge Painting Rust Defects and Their Limitations. Proceedings of the 2005 ASCE International Conference on Computing in Civil Engineering, Cancun, Mexico. Lee, S., L.-M. Chang, et al. (2006). "Automated Recognition of Surface Defects Using Digital Color Image Processing." Automation in Construction 15: 540-549. Leu, S.-S. and S.-L. Chang (2005). "Digital Image Processing-Based approach for tunnel Excavation Faces." Automation in Construction 14: 750-765. Leung, S.-W., S. Mak, et al. (2008). "Using a Real-Time Integrated Communication System to Monitor the Progress and Quality of Construction Works." Automation in Construction 17: 749-757. Lukins, T. C., Y. M. Ibrahim, et al. (2007). Now You See It:The Case for Measuring Progress with Computer Vision. Proceedings of the 4th International SCRI Research Symposium, Salford UK. Lukins, T. C. and E. Trucco (2007). Towards Automated Visual Assessment of Progress in Construction Projects. Proceedings of the British Machine Vision Conference, Warwick, UK. Memon, Z. A., M. Z. Abd.Majid, et al. (2005). An Automatic Project Progress Monitoring Model by Integrating Auto CAD and Digital Photos. Proceedings of the 2005 ASCE International Conference on Computing in Civil Engineering, Cancun, Mexico. Memon, Z. A., M. Z. AbdMajid, et al. (2006). Investigating the issue of project progress performance and proposing a prototype software: A case study of the Malaysian construction industry. Joint International conference on computing and decision making in civil and building engineering, Montreal, Canada. Miah, T., C. Carter, et al. (1998). "Wearable Computers - An Application of BT's Mobile Video System for the Construction Industry." BT Technol Journal 16(1): 191-199. Navon, R. (2000). "Process and quality control with a video camera, for a floor-tiling robot." Automation in Construction 10: 113-125. Nuntasunti, S. and L. E. Bernold (2002). Beyond WEBCAM: A site-Web-Site for Building Construction. International Symposium on Automation and Robotics in Construction, 19th (ISARC) Proceedings, Gaithersburg, Maryland.
115

The Built & Human Environment Review, Volume 1, 2008

Obaidat, M. T. and S. A. Al-kheder (2006). "Integration of Geographic Information Systems and Computer Vision Systems for Pavement Distress Classification." Construction and Building Materials 20: 657-672. Ordonez, C., P. Arias, et al. (2008). "Two Photogrammetric Methods for Measuring Flat Elements in Buildings Under Construction." Automation in Construction 17: 517-525. Paterson, A. M., G. R. Dowling, et al. (1997). "Building Inspection: can computer vision help?" Automation in Construction 7: 13-20. Podbreznik, P. and D. Rebolj (2005). Automatic Comparison of Site Images and the 4D Model of the Building. 22nd W78 International Conference on Information Technology in Construction, Dresden, Germany. Podbreznik, P. and D. Rebolj (2006). Building Elements Recognition Using Site Images and 4D Model. Joint International Conference on Computing and Decision Making in Civil and Building Engineering, Montreal, Canada. Podbreznik, P. and D. Rebolj (2007). Real-Time activity Tracking System - The Development Process. 24th W78 Conference: Bringing ITC Knowledge to Work, Maribor, Slovenia. Quinones, C. A., Y. M. A. Hashash, et al. (2008). "Digital Image Reasoning for Tracking Excavation Activities." Automation in Construction 17: 608-622. Saad, I. M. H. and D. E. Hancher (1998). "Multimedia for Construction Project Management: Project Navigator." Journal of Construction Engineering and management 124(1): 82-89. Schofield, A. J., T. J. Stonham, et al. (1997). "Automated People Counting to Aid Lift Control." Automation in Construction 6: 437-445. Shih, N.-J., J.-T. Lai, et al. (2006). "The Application of a Panorama Image Database Management System (PIDMS) for Information Integration on Construction Sites." ITcon 11: 641-654. Trucco, E. and A. P. Kaka (2004). "A Framework for Automatic Progress Assessment on Construction Sites Using Computer Vision." International Journal of IT in Architecture, Engineering and Construction 2(2): 147164. Trucco, E. and A. Verri (1998). Introductory Techniques for 3-D Computer Vision. Upper Saddle River, New Jersey, Prentice Hall. Trupp, T., C. Marulanda, et al. (2004). Novel Technologies for Tracking Construction Progress of Deep Excavations. Geotechnical Engineering for Transportation Projects (GSP No. 126), Los Angeles, California, USA. Tung, P.-C., Y.-R. Hwang, et al. (2002). "The Development of a Manipulator Imaging System for Bridge Crack Inspection." Automation in Construction 11: 717-729. Umetani, T., Y. Mae, et al. (2007). "Automated identification of Partially Exposed Metal Object." Automation in Construction 16: 842-851. Viola, P. and M. Jones (2001). Rapid object detection using a boosted cascade of simple features. IEEE Conference on Computer Vision and Pattern Recognition. Wu, Y. and H. Kim (2004). Digital Imaging in Assessment of Construction Project Progress. Proceedings of the 21st International Symposium on Automation and Robotics in Construction, Korean institute of Construction Technology (KICT), Jeju, Korea.

116

The Built & Human Environment Review, Volume 1, 2008

Yu, S.-N., J.-H. Jang, et al. (2007). "Auto Inspection System Using a Mobile Robot for Detecting Concrete Cracks in a Tunnel." Automation in Construction 16: 255-261. Zhu, Z. and I. K. Brilakis (2008). "Detecting Air Pockets for architectural Concrete Quality Assessment Using Visual Sensing." ITcon 13(Special Issue on Sensors in Construction and Infrastructure Management): 86-102. Zou, J. and H. Kim (2007). "Using Hue, Saturation, and Value Color Space for Hydraulic Excavator Idle Time Analysis." Journal of Computing in Civil Engineering 21(4): 238-246.

117