Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Bridge Inspection and condition assessment using Unmanned Aerial Vehicles (UAVs): Major challenges and solutions from a practical perspective

  • Jung, Hyung-Jo (Department of Civil and Environmental Engineering, Korean Advanced Institute for Science and Technology) ;
  • Lee, Jin-Hwan (Department of Civil and Environmental Engineering, Korean Advanced Institute for Science and Technology) ;
  • Yoon, Sungsik (Department of Civil and Environmental Engineering, Korean Advanced Institute for Science and Technology) ;
  • Kim, In-Ho (Department of Civil and Environmental Engineering, Korean Advanced Institute for Science and Technology)
  • Received : 2019.06.19
  • Accepted : 2019.08.29
  • Published : 2019.11.25
⟨ Previous Next ⟩

Abstract

Bridge collapses may deliver a huge impact on our society in a very negative way. Out of many reasons why bridges collapse, poor maintenance is becoming a main contributing factor to many recent collapses. Furthermore, the aging of bridges is able to make the situation much worse. In order to prevent this unwanted event, it is indispensable to conduct continuous bridge monitoring and timely maintenance. Visual inspection is the most widely used method, but it is heavily dependent on the experience of the inspectors. It is also time-consuming, labor-intensive, costly, disruptive, and even unsafe for the inspectors. In order to address its limitations, in recent years increasing interests have been paid to the use of unmanned aerial vehicles (UAVs), which is expected to make the inspection process safer, faster and more cost-effective. In addition, it can cover the area where it is too hard to reach by inspectors. However, this strategy is still in a primitive stage because there are many things to be addressed for real implementation. In this paper, a typical procedure of bridge inspection using UAVs consisting of three phases (i.e., pre-inspection, inspection, and post-inspection phases) and the detailed tasks by phase are described. Also, three major challenges, which are related to a UAV's flight, image data acquisition, and damage identification, respectively, are identified from a practical perspective (e.g., localization of a UAV under the bridge, high-quality image capture, etc.) and their possible solutions are discussed by examining recently developed or currently developing techniques such as the graph-based localization algorithm, and the image quality assessment and enhancement strategy. In particular, deep learning based algorithms such as R-CNN and Mask R-CNN for classifying, localizing and quantifying several damage types (e.g., cracks, corrosion, spalling, efflorescence, etc.) in an automatic manner are discussed. This strategy is based on a huge amount of image data obtained from unmanned inspection equipment consisting of the UAV and imaging devices (vision and IR cameras).

Keywords

Acknowledgement

Supported by : Ministry of Land, Infrastructure and Transport (MOLIT)

References

  1. Akbar, M.A., Qidwai, U. and Jahanshahi, M.R. (2019), "An evaluation of image-based structural health monitoring using integrated unmanned aerial vehicle platform", Struct. Control Health Monit., 26, e2276. DOI: 10.1002/stc.2276.
  2. American Society for Civil Engineers (ASCE) (2017), https://www.infrastructurereportcard.org/.
  3. Brooks, C., Dobson, R.J., Banach, D.M., Dean, D., Oommen, T., Wolf, R.E. and Hart, B. (2015), "Evaluating the use of unmanned aerial vehicles for transportation purposes (No. RC-1616)", Michigan Tech Research Institute, Ann Arbor. https://www.micigan.gov/documents/mdot/RC1616_Part_C_488515_7.pdf
  4. Chan, B., Guan, H., Jo, J. and Blumenstein, M. (2015), "Towards UAV-based bridge inspection systems: a review and an application perspective", Struct. Monit. Maint., 2(3), 283-300. https://doi.org/10.12989/smm.2015.2.3.283.
  5. Chen, S., Linh, T.-H., Laefer, D. and Mangina, E. (2018), "Automated bridge deck evaluation through UAV derived point cloud", Proceedings of the 2018 Civil Engineering Research in Ireland Conference, Dublin, August.
  6. Chen, S., Laefer, D.F., Mangina, E., Zolanvari, S.M.I. and Byrne, J. (2019), "UAV bridge inspection through evaluated 3D reconstructions", J. Bridge Eng., 24(4), 05019001. DOI: 10.1061/(ASCE)BE.1943-5592.0001343.
  7. Cho, S. and Lee, S. (2009), "Fast Motion Deblurring", ACM Trans. Graph., 28(5), Article 145. DOI: 10.1145/1618452.1618491.
  8. Darby, P., Hollermann, W. and Miller, J. (2019), "Exploring the potential utility of unmanned aerial vehicles for practical bridge inspection in Louisiana", Proceedings of the MATEC Web of Conferences, 271, 01001. DOI: 10.1051/matecconf/201927101001.
  9. Delgado, J.P., Soria, P.R., Arrue B.C. and Ollero, A. (2017), "Bridge mapping for inspection using an UAV assisted by a total station", Proceedings of the ROBOT 2017: Third Iberian Robotics Conference, 309-319. DOI: 10.1007/978-3-319-70836-2_26.
  10. Dorafshan, S. and Maguire, M. (2018), "Bridge inspection: human performance, unmanned aerial systems and automation", J. Civil Struct. Health Monit., 8, 443-476. DOI: 10.1007/s13349-018-0285-4.
  11. Duque, L. (2017), "UAV-based bridge inspection and computational simulations", Theses and Dissertations, 2159, South Dakota State University, South Dakota.
  12. Eschmann, C., Kuo, C.M., Kuo, C.H. and Boller, C. (2013), "High-resolution multisensor infrastructure inspection with unmanned aircraft systems", Proceedings of the International Archives of the Phtogrammetry, Remote Sensing and Spatial Information Sciences, Rostock, Germany, September.
  13. Fujino, Y. and Takada, T. (2008), "Risk assessment, management and monitoring of infrastructure systems", Department of Civil Engineering Report, University of Tokyo, Tokyo.
  14. Hallermann, N. and Morgenthal, G. (2014), "Visual inspection strategies for large bridges using Unmanned Aerial Vehicles (UAV)", Proceedings of the 7th International Conference on Bridge Maintenance, Safety and Management (IABMAS 2014), Shanghai, July.
  15. Han, K., Lin, J. and Golparvar-Fard, M. (2015), "A formalism for utilization of autonomous vision-based systems and integrate project models for construction progress monitoring", Proceedings of the Conference on Autonomous and Robotic Construction of Infrastructure, Ames, IA, USA, June.
  16. Hoskere, V., Narazaki, Y., Hoang, T.A. and Spencer, B.F. Jr. (2017), "Vision-based structural inspection using multiscale deep convolutional neural networks", Proceedings of the 3rd Huixian International Forum on Earthquake Engineering for Young Researchers, Urbana, IL, USA, August.
  17. Irizarry, J. and Johnson, E.N. (2014), "Feasibility study to determine the economic and operational benefits of utilizing unmanned aerial vehicles (UAVs)", FHWA-GA-1H-12-38. Georgia Institute of Technology. https://smartech.gatech.edu/handle/1853/ 52810?show=full.
  18. Jahanshahi, M.R., Kelly, J.S., Masri, S.F. and Sukhatme, G.S. (2009), "A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures", Struct. Infrastruct. Eng., 5(6), 455-486. DOI: 10.1080/15732470801945930.
  19. Jahanshahi, M.R. and Masri, S.F. (2013), "A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation", Smart Mater. Struct., 22(3), 035019. DOI: 10.1088/0964-1726/22/3/035019.
  20. Jang, K. and An, Y.K. (2018), "Multiple crack evaluation on concrete using a line laser thermography scanning system", Smart Struct. Syst., 22(2), 201-207. https://doi.org/10.12989/sss.2018.22.2.201.
  21. Jordan, S., Moore, J., Hovet, S. Box, J., Perry, J., Kirsche, K., Lewis, D. and Tse, Z.T.H. (2018), "State-of-the-art technologies for UAV inspections", IET Radar Sonar Nav., 12(2), 151-164. DOI: 10.1049/iet-rsn.2017.0251.
  22. Jung, S., Song, S., Kim, S., Park, J., Her, J., Roh, K. and Myung, H. (2019), "Toward autonomous bridge inspection: a framework and experimental results", Proceedings of the 16th International Conference on Ubiquitous Robots (UR), Jeju, Korea, June.
  23. Khaloo, A., Lattanzi, D., Cunningham, K., Dell'Andrea, R. and Riley, M. (2018), "Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling", Struct. Infrastruct. Eng., 14(1), 124-136. DOI: 10.1080/15732479.2017.1330891.
  24. Kim, H.J., Lee, J.H., Ahn, E.J., Cho, S.J., Shin, M.S. and Sim, S.H. (2017), "Concrete crack identification using a UAV incorporating hybrid image processing", Sensors, 17, 2052. DOI: 10.3390/s17092052.
  25. Kim, B. and Cho, S. (2019), "Image-based concrete crack assessment using mask and region-based convolutional neural network", Struct. Control Health Monit., DOI: 10.1002/stc.2381.
  26. Lee, Y., Kim, B. and Cho, S. (2018), "Image-based spalling detection of concrete structures incorporating deep learning", Proceedings of the 7th World Conference on Structural Control and Monitoring, Qingdao, China, July.
  27. Lovelace, B. and Zink, J. (2015), "Unmanned Aerial Vehicle Bridge Inspection Demonstration Project", Report No. MN/RC 2015-40, Minnesota Department of Transportation Research Services & Library, St. Paul, MN.
  28. Moller, S. (2008), "CALTRANS Bridge inspection aerial robot", CA08-0182, Final report. Division of research and innovation, California Department of Transportation. Scaramento. www.dot.ca.gov/newtech/researchreports/reports/2008/08-0182.pdf
  29. Morgenthal, G. Hallermann, N., Kersten, J., Taraben, J., Debus, P., Helmrich, M. and Redehorst, V. (2019), "Framework for automated UAS-based structural condition assessment of bridges", Automat. Constr., 97, 77-95. DOI: 10.1016/j.autcon.2018.10.006.
  30. Munguia, R., Urzua, S., Bolea, Y. and Grau, A. (2016), "Vision-based SLAM system for unmanned aerial vehicles", Sensors, 16(3), 372. DOI: 10.3390/s16030372.
  31. Myeong, W.C. and Myung, H. (2019), "Development of a wall-climbing drone capable of vertical soft landing using a tilt-rotor mechanism", IEEE Access, 7(1), 4868-4879. DOI: 10.1109/ACCESS.2018.2889686.
  32. Myeong, W.C. (2019), "Development of a wall-climbing UAV using tilt-rotor and an auxiliary arm", Ph.D. Dissertation, KAIST, Daejeon, Korea.
  33. Otero, L.D., Gagliardo, N., Dalli, D., Huang, W.H. and Cosentino, P. (2015), "Proof of concept for using unmanned aerial vehicles for high mast pole and bridge inspections", (No. BDV28 TWO977-02). Florida Institute of Technology. https://rosap.ntl.bts.gov/view/dot/29176/dot_29176_DS1.pdf
  34. Pan, J., Sun, D., Pfister, H. and Yang, M.H. (2016), "Blind image dublurring using dark channel prior", Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, June.
  35. Pereira, F.C. and Pereira, C.E. (2015), "Embedded image processing systems for automatic recognition of cracks using UAVs", IFAC-PapersOnLine, 48, 16-21. DOI: 10.1016/j.ifacol.2015.08.101.
  36. Sabatini, R., Gardi, A. and Richardson, M.A. (2014), "LIDAR obstacle warning and avoidance system for unmanned aircraft", Int. J. Comput. Syst. Eng. 8, 711-722. DOI: 10.1016/j.ast.2016.05.020.
  37. Sacks, R., Kedar, A., Borrmann, A., Ma, L., Brilakis, I., Huthwohl, P., Daum, S., Kattel, U., Yosef, R., Liebich, T., Barutch, B.E. and Muhic, S. (2018), "SeeBridge as next generation bridge inspection: overview, information delivery manual and model view definition", Automat. Constr., 90, 134-145. DOI: 10.1016/j.autcon.2018.02.033.
  38. Salaan, C.J.O., Okada, Y., Mizutani, S., Ishii, T., Koura, K., Ohno, K. and Tadokoro, S. (2018), "Close visual bridge inspection using a UAV with a passive rotating spherical shell", J. Field Robotics, 35, 850-867. DOI: 10.1002/rob.21781.
  39. Sieberth, T., Wackrow, R. and Chandler, J.H. (2016), "Automatic detection of blurred images in UAV image sets", ISPRS J Photogramm, 122, 1-16. DOI: 10.1016/j.isprsjprs.2016.09.010.
  40. Song, S., Jung, S., Kim, H. and Myung, H. (2018), "A method for mapping and localization of quadrotors for inspection under bridges using camera and 3D-LiDAR", Proceedings of the 7th Asia-Pacific Workshop on Structural Health Monitoring, Hong Kong SAR, P.R. China. November.
  41. Spencer, Jr., B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", J. Eng., 5, 199-222. DOI: 10.1016/j.eng.2018.11.030.
  42. Wells, J. and Lovelace, B. (2017), "Unmanned Aircraft System Bridge Inspection Demonstration Project Phase II", Report No. MN/RC 2017-18, Minnesota Department of Transportation Research Services & Library, St. Paul, MN.
  43. Wu, L. (2015), "Applications of computer vision technology of automated crack detection and quantification for the inspection of civil infrastructure systems", Ph.D. Dissertation, University of Central Florida.
  44. Yamada, M., Nakao, M., Hada, Y. and Sawasaki, N. (2017), "Development and field test of novel two-wheeled UAV for bridge inspections", Proceedings of the 2017 International Conference on Unmanned Aircraft Systems (ICUAS), Miami, FL, USA. June.
  45. Yeum, C.M. and Dyke, S.J. (2015), "Vision-based automated crack detection for bridge inspection", Comput. Civ. Infrastruct. Eng., 30(10), 759-770. DOI: 10.1111/mice.12141.

Cited by

  1. Debonding defect quantification method of building decoration layers via UAV-thermography and deep learning vol.28, pp.1, 2021, https://doi.org/10.12989/sss.2021.28.1.055