Artificial intelligence, drones and sensors set to save our crumbling infrastructure

A version of this post originally appeared on the Inside IBM Research blog on Medium.

Infrastructure fails. The buzz about flying taxis, futuristic high-speed vacuum tunnels and even autonomous cars apart, in real life essential infrastructure is aging around the world, with bridges, motorways, dams and tunnels cracking at the seams. It’s a challenge that exceeds the capacity of any maintenance department.

However, new artificial intelligence technology from IBM Research and partners combined with data collected from drones and sensors can help save the day — and patch the cracks in time. In the United States, one in 13 bridges is considered structurally deficient — which means it has a significant defect that needs urgent repairs or at the very least less weight and reduced traffic speed. That’s 47,052 bridges out of the country’s 616,087 — and, according to a recent report, Americans zoom across these failing bridges 178 million times every single day.

And it’s not just bridges — concrete cracks on motorways and potholes impact the smooth goods delivery, affecting the economy. Tunnels collapsing can trap traffic inside and stay unusable for months or years, leading to congestion on nearby roads, and dams failing can trigger floods.

But spotting cracks in time on miles and miles of highways and countless bridges is not just costly and time consuming, but plain tricky.

Enter AI — and drones.

Drones take pictures, lots of them. While it can take a human team a month to examine a pillar of a large bridge (and they have to do it while dangling precariously using ropes and harnesses), a drone can zip around in just a day. Usually, drones are controlled by a human operator — but IBM researchers in the company’s Haifa lab in Israel have developed an automated navigation system that enables drones to fly totally autonomously.

Once programmed, they navigate around a structure and snap pictures of cracks. An AI system developed by IBM scientists in Zurich then analyzes the images to spot any potential problems and alert humans to investigate up close.

Apart from saving time and making the inspection process safer, another key advantage is that the drones take pictures always from the same angle and distance, instead of different people taking photos with a different perspective. This way, it’s then possible to compare the pictures, inspection after inspection, and monitor the progress of cracks over time.

But it’s not just about the drones. IBM is collaborating with Sund & Bælt, the owner and operator of some of the largest infrastructure in the world, and with Sacertis, an Italian firm that has developed a sensor-based monitoring system for bridges, roads and other infrastructure. The three firms are working jointly using several technologies: the drones that automatically collect high-resolution images at regular intervals, the AI for image interpretation and document understanding— and sensors inserted in structures to spot internal problems.

Using IBM’s AI, Sacertis has recently analyzed the images a drone took of Europe’s longest suspension bridge, Storebælt. It’s the third-longest suspension bridge in the world, linking the eastern and western parts of Denmark. The software hasn’t flagged any major defects that need urgent attention, says Andrea Cuomo, the founder of Sacertis, but it did find cracks.

“Cracks are intrinsic in concrete, and there is probably no concrete structure in the world that doesn’t have them,” he says. “But there are cracks you don’t care about and there are structural cracks undermining the stability. It means that the earlier you close the cracks, the better — as over time, humidity can get in and undermine your structure.”

With Storebælt images, the AI has been able to differentiate between cracks, an algae, and rust with 94 percent accuracy, says Dr. Cristiano Malossi, an IBM AI scientist leading the team that has developed NeuNetS, the platform for AI Automation. The accuracy “could potentially improve over time when we will get more data — we expect to have better AI models in the future,” he adds.

Integrating this approach with mathematical models, sensor-based monitoring systems, data analytics and advanced civil engineering knowledge allows the structural engineer to assess the stability of the structure much more precisely — as well as to estimate the life span and provide the necessary data to plan predictive maintenance.

The next step is to train the AI to differentiate between critical cracks and non-urgent ones. Over time, concrete ages — and many structures were built half a century ago or longer, designed for lighter loads than we have today. The software needs to analyze the stiffness, the safety of the structure and the load — and assess the risk. The IBM’s AI can’t yet determine which cracks are critical and which ones are cosmetic — at the moment, humans need to make the call. But the tech will get there soon, says Cuomo. “We will train the machine to understand what is critical and what is not, what was the structure and how was it built.”

Sensors and image recognition can help set the right priorities in future interventions — and monitoring digital twins for years is arguably easier than trying to survey every single bridge. After all, Cuomo says, “you can’t repair 1000 bridges all at once, or you’ll close the country.”