Robot with LiDAR laser explores danger zones

Robot systems explore unfamiliar terrain, buildings or danger zones with cameras. In the 3D-InAus project, researchers from the Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE are using a LiDAR laser on a mobile robot, emitting laser pulses to measure distances. The results are used to produce geometrically accurate 3D environments.

In a disaster such as a chemical plant incident or flooding, emergency services need ways to quickly get an overview of the situation. But in many cases, they are not permitted to enter the scene itself in order to avoid putting themselves at risk.

Researchers at Fraunhofer FKIE in Wachtberg are working on a solution to this problem in the 3D-InAus project. A robot equipped with a LiDAR (light detection and ranging) laser explores the area. LiDAR technology uses pulses of light to scan the environment and measure distances. This creates a 3D model showing buildings, rooms, open space, objects and all of the associated dimensions and distances. Users can move freely around the 360-degree visualization using a joystick as they investigate the virtual environment.

Timo Röhling, a technical project manager from the Cognitive Mobile Systems department, explains, “Compared to robot systems that use cameras to explore a danger zone, our project goes a big step farther. The laser pulses supply measurements for precision 3D cartography of an area of terrain or building. Distances and dimensions are not estimated but instead determined with accuracy down to just a few centimeters.”

Geometric point cloud derived from laser pulses

The centerpiece of the hardware is a LiDAR laser mounted on a turntable. A rotating mirror is built into the LiDAR module. It can scan an area made up of 16 vertical sections, or “slices,” 10 times per second. The turntable rotates the laser so that the vertical sections cover an entire 360-degree view along the horizontal axis.

The system generates a total of 1.3 million laser pulses per second. These pulses bounce off the surrounding objects, and the time lag in between is used to calculate the relevant distance. The LiDAR module is mounted on a vehicle that moves through the area, either continuously or in stop-and-go mode.

Continuous operation is much faster but also less accurate. The result is a 3D point cloud in which each point stands for a laser pulse or distance measurement. There is also a camera system with up to six cameras. The images from the cameras are used to color the associated objects or shapes.

“You might think of us melding the camera images and point cloud together. This gives us a vivid, detailed and also geometrically accurate 3D environment showing buildings, open space and objects,” Röhling says.

The raw data acquired by the LiDAR laser is pre-processed by a computer module inside the robot before it even finishes its mission. The final visualization is then produced on a stationary basis during post-processing. It takes about three hours to map an area measuring 400 x 400 m. In a disaster when time is of the essence, accelerated operation can be used to get an initial overview in as little as one hour. It is also possible to explore an area using multiple vehicles simultaneously.

This method of 3D mapping is also valuable to the Bundeswehr, which commissioned the research project. It can be used to generate complex situational overviews of unfamiliar terrain or a danger zone, and thus save service members’ lives. The system’s software is able to process the measurement values from sensors that detect gaseous toxic substances or sources of radiation and then place them on the 3D maps.

Virtual GPS inside the building

The robot system is generally radio-controlled by a user working with a joystick and tablet. If there is no radio contact, the robot systems could also move around the terrain autonomously.

Exploring buildings is a challenge, as there is no GPS reception there, but the researchers at Fraunhofer FKIE have found a solution for this as well. The building’s position and size are already known from the mapping of the terrain, and the software uses that information to generate a virtual GPS for the interior of the building. That allows the robot system to navigate autonomously inside the structure as well.

The team from Fraunhofer FKIE drew on their years of expertise in robot-assisted modeling of 3D environments for their work on the project. “We came up with the concept, selected the components and implemented the algorithms,” Röhling explains.

The team was careful to ensure that the robot system would be as versatile as possible. The laser module and turntable can be mounted on a wide range of different vehicles, for example. Depending on the terrain, vehicles with wheels or tracks or even drones could be used. Users assemble the specific components to fit each scenario.