GAAS
  • Hello Sky
  • Build Your Own Autonomous Drone: Software Realization
    • An Overview of GAAS
    • GAAS 总览
    • Prerequisite handbook for GAAS: Linux tutorial
    • 先修知识速查手册:Linux操作
    • Autonomous Drone Software E01: OFFBOARD Control and Gazebo Simulation
    • 无人机自动驾驶软件系列 E01:OFFBOARD控制以及Gazebo仿真
    • Autonomous Drone Software E02 : build a 3D model with your drone
    • 无人机自动驾驶软件系列 E02 : 通过无人机机载摄像头构建建筑物 3D 模型
    • Autonomous Drone Software E03: Using SLAM In GPS Denied Environment For Position Estimation
    • 无人机自动驾驶软件系列 E03: 在无GPS环境下通过SLAM实现位置估计
    • Autonomous Drone Software E04: Depth Estimation, Octomap and Path Planning
    • 无人机自动驾驶软件系列 E04 : 深度估计,八叉树地图以及路径规划
    • Autonomous Drone Software E05: Vision-Based Auto-Landing
    • 无人机自动驾驶软件系列 E05: 视觉引导降落
    • Autonomous Drone Software E06: Basic Object Tracking
    • 无人机自动驾驶软件系列 E06: 简单全局目标追踪
  • Build Your Own Autonomous Drone: Hardware Implementation
    • Autonomous Drone Hardware E01: Building a Drone from Scratch
    • 无人机自动驾驶硬件系列 E01:从 0 开始组装测试用无人机
    • Autonomous Drone Hardware E02: Step-by-Step Flight Control Setup with QGroundControl
    • 无人机自动驾驶硬件系列 E02:手把手教学 QGroundControl 下的飞控校准
  • Case Study
    • 在 AirSim 中模拟无人机对客机巡检
    • Using AirSim to Simulate Aircraft Inspection by Autonomous Drones
  • Handy Tools
    • GAAS v0.7 Release Mirror - x64
    • GAAS v0.7 Release 镜像 - x64
    • GAAS Mirror for TX2 (beta)
    • GAAS-TX2 镜像 (beta)
    • GAAS prebuilt dependencies
    • GAAS依赖包预编译版
    • 系列视频
Powered by GitBook
On this page
  • Environment Update
  • Start Simulation
  • Record A Rosbag
  • Process Rosbag
  • Building the Model

Was this helpful?

  1. Build Your Own Autonomous Drone: Software Realization

Autonomous Drone Software E02 : build a 3D model with your drone

In this tutorial, we will talk about a real world use case on how to build a 3D model of a building based on what we have learned in the last tutorial.

Previous无人机自动驾驶软件系列 E01:OFFBOARD控制以及Gazebo仿真Next无人机自动驾驶软件系列 E02 : 通过无人机机载摄像头构建建筑物 3D 模型

Last updated 5 years ago

Was this helpful?

Give us a Star on if you find this tutorial useful.

Help us to understand what you want to learn in the upcoming tutorials by fill out this .

---------------------

We suggest developers to install GAAS mirror to use the project:

Should you have any questions or suggestions to the tutorials or GAAS itself, let us know and join our

In the last tutorial we walked through how to control your drone with MAVROS in OFFBOARD mode within Gazebo simulation environment. Now in this tutorial, we will walk you through the process to build a 3D modell using the same API as what we used last time and a website (It's free!) for building models.

This tutorial might be off the topic and have little to do with Autonomous Drone, but building a 3D model with drone has been adopted by many companies, you can check this link:

https://www.archdaily.com/899196/how-to-use-a-drone-to-create-a-detailed-3d-context-model

and by building a 3D model, we have the chance to realize more and more complicated tasks with simple APIs.

Steps required to build such a model are:

  1. launch Gazebo simulation environment;

  2. takeoff the drone and record a rosbag;

  3. command the drone to fly a circle around a typical building;

  4. land the drone and process images from rosbag;

  5. upload images to provided website.

Or if you wish to build a model about a real building or an environment:

  1. takeoff the drone with your controller or you can choose to use provided Python API if you have stable SSH access to your companion computer (take extreme care);

  2. start recording a rosbag;

  3. command the drone to fly around the building and make sure the building or environment is thoroughly covered by the camera;

  4. land the drone and process the images from rosbag;

  5. upload images to provided website.

We will walk you through each of the above step in Gazebo simulation environment and we will provide an website that you can use to build a model. After this tutorial, you will build something like this all by yourself!

Environment Update

Before starting, remember to update GAAS:

# change directory to where you put GAAS and git pull from master branch to update
cd (GAAS_PATH)
git pull origin master

Next, copy files that are required by Gazebo simulation to where you put PX4 firmware by:

cp -r (GAAS_PATH)/simulator/launch/* ~/catkin_ws/src/Firmware/launch/
cp -r (GAAS_PATH)/simulator/models/* ~/catkin_ws/src/Firmware/Tools/sitl_gazebo/models/
cp -r (GAAS_PATH)/simulator/worlds/* ~/catkin_ws/src/Firmware/Tools/sitl_gazebo/worlds/
cp -r (GAAS_PATH)/simulator/posix-config/* ~/catkin_ws/src/Firmware/posix-configs/SITL/init/ekf2/

After completing, you can continue to the next step.

Start Simulation

If you followed the last tutorial and the previous step you will have everything we need to launch the simulation. To launch the simulation, in a terminal:

roslaunch px4 sfm.launch

it will launch MAVROS as well as Gazebo simulation at the same time and your drone will be centered at the origin:

While running Gazebo, if everything works properly with the exception of the above Err message, the Err message itself DOES NOT affect the performance of Gazebo.

If you happen to know how to resolve this Err message, please submit a PR on GitHub to help us make it go away.

Before you continue, remember to check MAVROS connection status by:

# make sure "connected" is "True"
rostopic echo /mavros/state

When you started simulation, sensor data such as IMU, barometer, camera and GPS etc will be published via MAVROS, a list of available topics can be checked by:

rostopic list

And some of the topics published by MAVROS are:

/gi/simulation/left/image_raw
/gi/simulation/right/image_raw
/mavros/imu/diff_pressure
/mavros/imu/mag
...

What we will be using for reconstructing the 3D model is only the left camera image:

/gi/simulation/left/image_raw

you can view images published from this topic using RVIZ by:

rviz

You will see a window which looks like this:

in the left bottom, select 'Add' and followed by 'by topic' and select '/gi/simulation/left/image_raw' you will see the current left camera image in real time. We will be building a 3D model in this Gazebo environment.

Record A Rosbag

Rosbag is like a box that wraps data published from ROS/MAVROS, and to build a 3D model, we need a series of images about a building or an environment taken from different angles. To take images and store them, we can use Rosbag.

Since we will only be using the left camera information we only need to subscribe to the left camera topic, and in order to do this, in a terminal:

rosbag record /gi/simulation/left/image_raw -O sfm.bag

Next, takeoff the drone:

python  (GAAS_PATH)/demo/tutorial_2/2_Struction_from_Motion/px4_mavros_run.py

The drone will take off to 2 meters high. Now we can start controlling the drone to move around the house to take pictures, while doing this we need to keep the drone headed towards the house so that we can capture detailed information about the house:

python (GAAS_PATH)/demo/tutorial_2/fly_circle.py

The drone will fly around the house at 2 meters high and finally it will arrive at the origin and land on the ground.

While recording the bag, we should keep the following points in mind in order to build a good model of the building:

  1. try to take images of a building where images are partially "overlapping" with consecutive ones;

  2. avoid sudden yaw change while taking images;

  3. try to cover the entire building, sometimes multiple circles are required to cover taller buildings.

You've probably noticed that we commanded the drone to fly a circle rather than a square, which seems more easy as much fewer waypoints would be needed. The reason behind this is a square usually means a big yaw change (90 degrees at each corner) while taking pictures and it is what need to avoid according to the second point mentioned above.

If the drone landed, you should stop rosbag recording using "ctril + c"

Now you will have a rosbag named 'sfm.bag'. Next, let's process this bag into a series of images before starting to build the model.

Process Rosbag

To build a 3D model, we need images rather than a rosbag. To process the rosbag into a series of images, you can use provided python scripts in the tutorial_2 folder:

python bag2image.py --bag (PATH-TO-YOUR-BAG) --output_path (IMAGE-OUTPUT-FOLDER) --image_topic /gi/simulation/left/image_raw

After this step, you will extract all the images in the bag to your local folder.

Try to limit image number, small scene would usually require less than 100 images. Sometimes you might need to remove redundant images.

Building the Model

We provided a website to build the model, and all you need to do is to upload images. Due to limited computation power, it might take a while before you can get the model.

aviation.giai.tech

Following is a real world use cases made by the website.

After uploading images to the above website (cannot upload image folder), you will need to wait for a while. Once the model is generated successfully you will receive an email and you can fetch the model in the provided website.

If you find any bug or need more help, please let us know either by or through our .

Give us a Star on if you find this tutorial useful.

Help us to understand what you want to learn in the upcoming tutorials by fill out this .

🌟
Github
three-question survey
GAAS v0.7 Release Mirror - x64
slack
opening an issue
facebook group
🌟
Github
Support us on Patreon.
three-question survey
Fig 1, 3D model
Fig 2, Gazebo simulation environment
Fig 3, RVIZ
Fig 4, select image topic
Fig 5, image display in Rviz
Fig 6, 3D reconstruction website
Fig 6, ChangChun JinYing Square built with 76 images (human controlled flight)