TY - JOUR

T1 - Automated damage detection for port structures using machine learning algorithms in heightfields

AU - Hake, Frederic

AU - Lippmann, Paula

AU - Alkhatib, Hamza

AU - Oettel, Vincent

AU - Neumann, Ingo

N1 - Funding Information: Open Access funding enabled and organized by Projekt DEAL. This research was funded by German Federal Ministry of Transport and Digital Infrastructure grant number 19H18011C. Funding Information: This work was carried out as part of the joint research project “3DHydroMapper–Bestandsdatenerfassung und modellgestützte Prüfung von Verkehrswasserbauwerken.” It consists of five partners and one associated partner: Hesse und Partner Ingenieure (multisensor system and kinematic laser scanning), WK Consult (structural inspection, BIM, and maintenance planning), Niedersachsen Ports (sea and inland port operation), Fraunhofer IGP (automatic modelling and BIM), Leibniz University Hannover (route planning and damage detection), and Wasserstraßen- und Schifffahrtsverwaltung des Bundes (management of federal waterways).

PY - 2023/6

Y1 - 2023/6

N2 - Marine infrastructures such as harbours, bridges, and locks are particularly exposed to salt water and are therefore subject to increasing deterioration. This makes regular inspection of the structures necessary. The inspection is carried out manually, using divers under water. To improve this costly and time-consuming process, we propose to scan the surface and underwater structure of the port with a multi-sensor system (MSS) and classify the obtained point cloud into damaged and undamaged areas fully automatically. The MSS consists of a high-resolution hydro-acoustic underwater multi-beam echo-sounder, an above-water profile laser scanner, and five HDR cameras. In addition to the IMU-GPS/GNSS method known from various applications, hybrid referencing with automatically tracking total stations is used for positioning. The key research idea relies on 3D data from TLS, multi-beam or dense image matching. For this purpose, we build a rasterised heightfield of the point cloud of a harbour structure by reducing the CAD-based geometry from the measured 3D point cloud. To do this, we fit regular shapes into the point cloud and determine the distance of the points to the geometry. To detect anomalies in the data, we use two methods in our approach. First, we use the VGG19 Deep Neural Network (DNN), and second, we use the Local-Outlier-Factors (LOF) method. To test and validate the developed methods, training data was simulated. Afterwards, the developed methods were evaluated on real data set in Lübeck, Germany, which were acquired with the developed Multi-Sensor-System (MSS). In contrast to the traditional, manual method by divers, we have presented an approach that allows for automated, consistent, and complete damage detection. We have achieved an accuracy of 90.5% for the method. The approach can also be applied to other infrastructures such as tunnels and bridges.

AB - Marine infrastructures such as harbours, bridges, and locks are particularly exposed to salt water and are therefore subject to increasing deterioration. This makes regular inspection of the structures necessary. The inspection is carried out manually, using divers under water. To improve this costly and time-consuming process, we propose to scan the surface and underwater structure of the port with a multi-sensor system (MSS) and classify the obtained point cloud into damaged and undamaged areas fully automatically. The MSS consists of a high-resolution hydro-acoustic underwater multi-beam echo-sounder, an above-water profile laser scanner, and five HDR cameras. In addition to the IMU-GPS/GNSS method known from various applications, hybrid referencing with automatically tracking total stations is used for positioning. The key research idea relies on 3D data from TLS, multi-beam or dense image matching. For this purpose, we build a rasterised heightfield of the point cloud of a harbour structure by reducing the CAD-based geometry from the measured 3D point cloud. To do this, we fit regular shapes into the point cloud and determine the distance of the points to the geometry. To detect anomalies in the data, we use two methods in our approach. First, we use the VGG19 Deep Neural Network (DNN), and second, we use the Local-Outlier-Factors (LOF) method. To test and validate the developed methods, training data was simulated. Afterwards, the developed methods were evaluated on real data set in Lübeck, Germany, which were acquired with the developed Multi-Sensor-System (MSS). In contrast to the traditional, manual method by divers, we have presented an approach that allows for automated, consistent, and complete damage detection. We have achieved an accuracy of 90.5% for the method. The approach can also be applied to other infrastructures such as tunnels and bridges.

KW - Damage detection

KW - Infrastructure

KW - Laser scanning

KW - Machine learning

KW - Multibeam echo sounder

UR - http://www.scopus.com/inward/record.url?scp=85147700226&partnerID=8YFLogxK

U2 - 10.1007/s12518-023-00493-z

DO - 10.1007/s12518-023-00493-z

M3 - Article

VL - 15

SP - 349

EP - 357

JO - Applied Geomatics

JF - Applied Geomatics

IS - 2

T2 - 5th Joint International Symposium on Deformation Monitoring 2022

Y2 - 20 June 2022 through 22 June 2022

ER -