Using AirSim to Simulate Aircraft Inspection by Autonomous Drones
Last updated
Last updated
Regular maintenance of aircraft is critical to the safety of flights. However, due to the size of the aircraft, maintenance technicians are having trouble inspecting the top of the aircraft. Since 2017, Airbus has been using drones to help with the maintenance of aircraft. The use of drones allows maintenance technicians to collect a large number of pictures of the aircraft for safety analysis.
Existing drones are not suitable for such task. First, it requires pilots to operate the drone. A minor deviation from the planned flight route could damage the aircraft. Also, for effective inspection, the pictures of aircraft parts must be taken at the same angle every time. A lot of pilots are not qualified or willing to operate drones under such risky and challenging scenario. Second, traditional drones rely on GPS, but indoor aircraft storage does not have GPS reception.
This case study shows how to use GAAS to control autonomous drones for aircraft inspection. This case study is based on a real project currently ongoing in the GAAS team. Minor details are changed for confidentiality purpose.
AirSim is an open-source simulator for drones, cars and more, built on Unreal Engine and developed by Microsoft. Comparing to Gazebo, AirSim has a more realistic simulated environment. AirSim supports PX4 based Hardware-In-The-Loop (HITL) and Software-In-The-Loop (SITL)。
Like all Unreal project, AirSim projects can be packaged. However, I do not recommend you to do so under Ubuntu. Using PX4 SITL in AirSim under Ubuntu can cause the program to crash. We are working on fixing related issues.
Also, Unreal needs around 69GB hard disk space with relatively high computational resources. If you do not want to install such a heavy software on your computer, you may choose to do the simulation in Gazebo. The configuration for Gazebo is way easier.
In order to run GAAS along with AirSim, we will be using Unreal for Linux.
First, register at Epic Games and connect it to your GitHub account.
Compile Unreal 4.22. It may take around 69 GB hard disk space.
Note：AirSim supports Unreal 4.18, but the background used in this case study is only available in Unreal 4.22. Therefore, we will continue this case study in Unreal 4.22.
Clone and compile AirSim：
Under Linux environment, go to the Unreal directory and run UnrealEngine/Engine/Binaries/Linux/UE4Editor.
Now you can see the simulator interface. Under the AirSim directory Unreal/Environments, there is a default project named Blocks . Copy AirSim/Unreal/Enviroments/Plugins to AirSim/Unreal/Enviroments/Blocks . Then open the project in Unreal.
Before using PX4 in AirSim, we should install and launch PX4 SITL. Through PX4 SITL, we may use PX4 as the flight control for drones in AirSim.
First, make sure that you have installed all dependencies required by PX4 as listed in the [GAAS Tutorial] (https://gaas.gitbook.io/guide/)。
Install PX4 Firmware. Note: AirSim requires Firmware v1.8.2.
Compile and launch SITL mode：
You can edit
(Firmware_Path)/posix-configs/SITL/init/ekf2/iristo configure the settings for the drone
Edit AirSim Setting，The file can be found at ~/Documents/AirSim/settings.json。
In Unreal, click the Play button. The simulation is launched. You should see a drone in the simulator.
Press
F1key to see a list of shortcuts. You may also open FPS to view FPS at any time.
Now you may use QGC to control the basic movements of the drone.
GAAS uses MAVROS to communicate with PX4. First, we need to launch MAVROS. Please see GAAS Tutorial EP01 . We recommend using Method 2 for the installation of MAVROS.
PX4 uses UDP ports for communication, so we need to change the MAVROS fcu_url configuration. Start a new launch file with:
Change the fifth line <arg name="fcu_url" default="/dev/ttyACM0:57600" /> to <arg name="fcu_url" default="udp://:14540@localhost:14557" />。
Launch MAVROS：
Now, following the GAAS Tutorial EP01, you may control the drone with python scripts.
For the drone to fly without GPS, let's turn off GPS altogether. First, in PX4 SITL iris, change the vision position fusion in EKF2_AID_MASK to 8. Then, edit the (Firmware_Path)/posix-configs/SITL/init/ekf2/iris to change EKF2_AID_MASK from 1 to 8.
Now we need to set up the AirSim configuration to include the stereo camera. We have provided the configuration file in GAAS.
In this config file, GPS has been set to
false,and the stereo camera is included.
After the above steps, let's click the Play button to start the simulation. However, without GPS, the drone will not be able to take off. We need to turn on SLAM to allow the drone to operate in GPS-deficient environment.
We have set up a stereo camera with a baseline of 12cm in the configuration file. It is the same setup as GAAS SLAM. Before running SLAM algorithms, we need to publish the camera data from the simulator to ROS. Install the Python Library for Airsim and then under GAAS directory, simulator/AirSim , run drone_image_ros.py.
Next, follow GAAS Tutorial EP03 to use GAAS for inspection of aircraft.
Note: When using SLAM, go to
Edit >> Editor Preferencein Unreal to turn off theUse Less CPU when in Background, otherwise the frame rate will be affected.
Download QGroundControl, and change the listening port, as shown below:
First, build the aircraft storage background. We recommend purchasing background materials from the Unreal Marketplace. Purchasing and download must be done on Epic Games on Windows. Then edit the background to your preference with the help of the Unreal Documentation. Due to IP restrictions, we cannot share our Project file with everyone. If you encountered problems with building the background, we welcome you to ask your questions at AirSim repo or Unreal Forum.
