The PILOTING project aids in the deployment of Artificial Intelligence to enable critical infrastructure inspection and maintenance.
Infrastructures are ageing. This statement alone raises discussions and concerns regarding the ageing of infrastructures, encompassing considerations such as the identification of outdated infrastructure types, materials involved, and their consequential ageing effects. It delves into the criticality and severity of these effects, necessitating precise inspections and interventions.
The broader issue of infrastructure ageing is linked to the need for preventive maintenance or even replacement of infrastructures well before their designated design life expires, thereby exerting significant economic, environmental, and social impacts.
Material deterioration leading to alterations in the infrastructure’s intended purpose, which can pose safety threats, may result from various factors. These factors, whether controllable by human interventions or not, ideally require attention before damage occurs. The comprehensive list of causes encompasses earthquakes, geological factors, sustained or sudden loads exceeding the structure’s capacity, severe weather conditions (e.g., heavy rain, snow, or the effects of climate change), and unresolved maintenance issues over the infrastructure’s lifecycle (such as corrosion, water leakage, spalling, delamination, etc.).
Addressing the above remains crucial for maintaining the functionality, safety, and intended purpose of infrastructures. The examination of civil engineering structures involves a holistic evaluation of their overall or partial condition. This integrity assessment is of great importance as it enables the determination of their reliability under current or anticipated loads (physical or other), and identifies the corresponding intervention level and means required.
Today, legal mandates impose inspection and maintenance obligations on all critical structures (buildings, bridges, tunnels, factories, ports, power plants, and more). Given the extended operational lifespan of such structures, there is a growing demand for versatile systems capable of adapting to diverse operational needs and accommodating various structure types with differing monitoring requirements.
Presently, structural inspection is widely recognised as a method for diagnosing the health of a structure, considering the composition of its materials, and evaluating its overall or partial condition. This comprehensive assessment encompasses multiple parameters collectively contributing to defining the structure’s integrity.
In this report, we are summarising recent research results of INLECOM INNOVATION in the PILOTING (PILOTs for robotic INspection and maintenance Grounded) project using robotics for infrastructure inspection and maintenance.
Robotics and automation for civil infrastructure inspection and maintenance
Robotics and automation solutions offer evident benefits over manual inspections, mostly in enhancing safety for both workers and the environment. Inspection and maintenance tasks can be carried out with strongly decreased risks over human life (minimised human presence and exposure).
Deploying robotics or automation eliminates the necessity for direct human interaction among infrastructure workers, particularly in challenging conditions like hazardous or difficult-to-access areas such as confined spaces, elevated heights, or zones with toxic substances.
At the same time, it enables more efficient infrastructure inspection, including difficult-to-inspect points, more precise measurements, more precise inspection repeatability and results aggregation.
Laser scanning for tunnel deformation assessment
Deformation assessment in transportation tunnels is considered a parameter measuring any deviation in the original 3D shape of the tunnel as a result of serious events (seismic, material failure, surrounding forces etc.).
INLECOM developed an autonomous and robotics-driven approach for capturing tunnels’ internal shape and comparing it with previous inspection data, being able to measure and localise any deformation with millimetre accuracy. The solution is highly innovative as it enables quick and safe measurements with large scalability prospects being able to capture a whole tunnel in 3D, regardless of its size/length.
The technical approach followed included the full integration of a FARO 3D laser scanner on robotics vehicles that autonomously navigated through the whole tunnel and triggered the laser scanner at positions of interest or the whole series of required snapshots for a whole section capturing. After the data collection at various positions, the 3D models were compared with previous inspections to detect any deformation of the tunnel.
To accomplish that, ICP (Iterative Closest Point) approaches were used as the first and most important aspect of the analysis, including a proper (and correct/precise) co-registration of the different models (cloud points). The methodology was validated at the motorway tunnels of Egnatia Motorway in Greece and proved to be highly applicable to other infrastructures.
The innovative aspects of the approach include precision measurements, expedited tunnel modeling, human-less operation, automated modeling and aggregation, seamless application in extended tunnels, and automated 3D model comparison and defect reporting.
Artificial Intelligence and computer vision for defect detection in oil and gas refineries
Corrosion stands out as a major hurdle in industries and infrastructural sectors where pipes and vessels play pivotal roles. These components are susceptible to corrosion, leading to structural deterioration, safety risks, environmental issues, and financial setbacks.
Early detection of corrosion is imperative to mitigate its detrimental impacts. Conventional inspection methods, often manual and time-intensive, face constraints in accurately gauging the extent of corrosion, especially in expansive construction sites.
Advancements in computer vision, Artificial Intelligence, and Deep Learning have created a new era of automated corrosion detection, leveraging capabilities to automate the process of corrosion detection, ultimately enhancing the overall integrity of infrastructure. The imperative for precise and efficient corrosion detection on construction sites arises from the potential consequences of unaddressed degradation.
INLECOM effort was dedicated to YOLO real-time, object-detection algorithms due to their compact size models that proved ideal for real-time situations and on-device deployment scenarios.
Other approaches included MASKRCNN for addressing instance classifications and Swin transformers as a basis for the feature representation of images due to their efficiency and accuracy.
Convolutional neural network-based Deep Learning models have demonstrated promising results in successfully and accurately recognising corrosion. For example, VGG and ResNet (deep neural networks), with numerous layers of convolutional and pooling operations, followed by fully connected layers for the final output predictions, can be utilised to detect corrosion. In this use case, supervised learning approaches were used.
The main challenges in this task included the collection of the datasets, while state-of-the-art methods were implemented and trained to identify corrosion on pipes and inside tanks surpassing 70% IoU (Intersection over Union).
Despite the huge benefits of automatic corrosion detection using AI, research challenges include data quality and accuracy, algorithm selection and optimisation. The algorithms were validated at the Chevron Oronite site, located in the north of France in Gonfreville L’Orcher near the port of Le Havre.
Holistic inspection and monitoring platform
In the frame of the PILOTING components and orchestration, INLECOM was the responsible partner for the integration of the PILOTING infrastructure inspection and maintenance platform, a comprehensive architecture synthesising key elements designed to revolutionise inspection and maintenance processes.
A major component of this platform was the robotics autonomous functionalities, as software modules seamlessly integrated into robotic vehicles to boast advanced capabilities, enabling functions like secure navigation in complex environments being actually the driving force within the robotic engines.
A robot control station was also developed to standardise the operation and control of inspection robots with the aspiration of creating a universal control interface for an array of robotic vehicles, minimising training costs and offering flexibility in the planning of inspection operations.
Another pivotal layer was a data distribution and harmonisation layer targeting to harmonise the diverse data generated during inspections by different robotics systems. Closing, a data management system was developed providing direct access to the digital information produced during both ongoing and historical inspection processes.
Addressing a longstanding industry challenge, the system centralises data storage, harmonises information, and indexes it based on various criteria, such as assets or inspection missions.
INLECOM’s role was dedicated to the integration, validation, and testing of the solution to the different robotics systems of the integrated solution. All components were also validated and demonstrated in the project pilot use cases.
Solutions technology deployment and validation
Use case one: Infrastructure inspection in oil and gas refineries
The first use case is an oil and gas refinery, the Chevron Oronite, located in the north of France in Gonfreville L’Orcher. This is a live industrial facility, Europe’s largest additive manufacturing facility.
The plant processes hydrocarbon products, and some areas of the plant are rated as explosive atmosphere (ATEX) zones. The plant is a large industrial complex which offered multiple opportunities to test inspection procedures at height, making it an ideal candidate for the PILOTING project, addressing a wide range of inspection categories such as large structure inspection, inspection of pipework at height, internal vessel inspection, ground monitoring and manipulation of equipment.
Testing for all refinery work scopes was conducted in the northern part of the Oronite plant split over four main work areas: tanks, water treatment plant, decommissioned modules, and pressure vessels. In this environment, INLECOM developed, tested, and validated its AI algorithms for automatic corrosion detection in pipes, vessels, and tanks.
Use case two: Inspection in viaducts
Ferrovial was the end user representing viaduct inspection operations, and the experiments were approached in order to perform a technical evaluation based on the developments concluded in the first phases of the project. Tests focused on verifying whether the flying devices are capable of carrying out the three expected use cases, from a technical point of view.
For this, Ferrovial provided a key test area that allowed the drones to demonstrate their capabilities within the scope of those use cases. The location selected is Alora, a small village in the interior of Malaga, through which the Spanish high-speed train runs, the Arroyo del Espinazo viaduct.
Use case three: Inspection in motorway tunnels
This pilot was driven by INLECOM in cooperation with Egnatia Motorway as the motorway operator and owner. The part of the Egnatia motorway selected for these experiments was the tunnel of Metsovo (in Ioannina), and a drainage tunnel very close to this.
These two tunnels were selected as i) the Metsovo tunnel indicated some interesting inspection points while is in full traffic and can represent traffic scenarios and realistic tunnel inspections, and ii) the drainage tunnel used as a reference tunnel without any traffic (so experiments could run in safety) while indicated a concrete tunnel without internal lining that could represent tunnels under construction.
INLECOM was particularly active in this use case, not only leading the piloting efforts but also testing the laser scanning approach for the tunnel deformation that was successfully validated in the final year.
The importance of robotics in infrastructure inspection
The domain of infrastructure inspection and maintenance is considered of primal interest and remains a recognised priority, particularly when talking about critical infrastructures.
Many industrial challenges are being currently tackled by recent research approaches and solutions, however, requirements relating to data availability, end-user and social acceptance, technological uptake and cost, as well as accuracy and precision of robotics and automation solutions remain to be solved.
INLECOM’s strategic roadmap and research vision are fully aligned with several components of the inspection and maintenance roadmap, including predictive maintenance systems, AI-based simulations and infrastructure Digital Twins, Augmented Reality solutions supporting inspection and maintenance operations, as well as AI-based defects detection and asset 3D deformation assessment.
About the team
The project partners form a competitive and multidisciplinary consortium consisting of 14 partners from eight EU countries, with strong competencies in automation and robotics as well as software and platform development tailored towards inspection and maintenance industrial requirements.
- Fundacion Andaluza Para El Desarrollo Aeroespacial – Spain
- Eidgenoessische Technische Hochschule Zuerich – Switzerland
- Universidad De Sevilla – Spain
- Chevron Oronite SAS – France
- Ferrovial Construccion SA – Spain
- Honeywell International SRO – Czech Republic
- Waygate Technologies Robotics AG – Switzerland
- Applus Rvis B.V. – Netherlands
- Egnatia Odos AE – Greece
- Quasset BV – Netherlands
- Sintef AS – Norway
- Inlecom Innovation Astiki Mi Kerdoskopiki Etaireia – Greece
- Institute Of Communication and Computer Systems – Greece
- Robotnik Automation SLL – Spain
This publication has been made following recent research results in the ‘PILOTING’, EC, co-funded, research project. This project has received funding from the European Union’s Horizon 2020 research and innovation program (No 871542-5). The authors acknowledge the research outcomes of this publication belonging to the PILOTING consortium.
