GAAS
  • Hello Sky
  • Build Your Own Autonomous Drone: Software Realization
    • An Overview of GAAS
    • GAAS 总览
    • Prerequisite handbook for GAAS: Linux tutorial
    • 先修知识速查手册:Linux操作
    • Autonomous Drone Software E01: OFFBOARD Control and Gazebo Simulation
    • 无人机自动驾驶软件系列 E01:OFFBOARD控制以及Gazebo仿真
    • Autonomous Drone Software E02 : build a 3D model with your drone
    • 无人机自动驾驶软件系列 E02 : 通过无人机机载摄像头构建建筑物 3D 模型
    • Autonomous Drone Software E03: Using SLAM In GPS Denied Environment For Position Estimation
    • 无人机自动驾驶软件系列 E03: 在无GPS环境下通过SLAM实现位置估计
    • Autonomous Drone Software E04: Depth Estimation, Octomap and Path Planning
    • 无人机自动驾驶软件系列 E04 : 深度估计,八叉树地图以及路径规划
    • Autonomous Drone Software E05: Vision-Based Auto-Landing
    • 无人机自动驾驶软件系列 E05: 视觉引导降落
    • Autonomous Drone Software E06: Basic Object Tracking
    • 无人机自动驾驶软件系列 E06: 简单全局目标追踪
  • Build Your Own Autonomous Drone: Hardware Implementation
    • Autonomous Drone Hardware E01: Building a Drone from Scratch
    • 无人机自动驾驶硬件系列 E01:从 0 开始组装测试用无人机
    • Autonomous Drone Hardware E02: Step-by-Step Flight Control Setup with QGroundControl
    • 无人机自动驾驶硬件系列 E02:手把手教学 QGroundControl 下的飞控校准
  • Case Study
    • 在 AirSim 中模拟无人机对客机巡检
    • Using AirSim to Simulate Aircraft Inspection by Autonomous Drones
  • Handy Tools
    • GAAS v0.7 Release Mirror - x64
    • GAAS v0.7 Release 镜像 - x64
    • GAAS Mirror for TX2 (beta)
    • GAAS-TX2 镜像 (beta)
    • GAAS prebuilt dependencies
    • GAAS依赖包预编译版
    • 系列视频
Powered by GitBook
On this page
  • Update Environment
  • Launch the Simulation
  • Build the Tracking Module
  • Testing
  • 1. Initialize the Simulation Environment
  • 2. Run the Tracking Algorithm
  • 3. Start Tracking

Was this helpful?

  1. Build Your Own Autonomous Drone: Software Realization

Autonomous Drone Software E06: Basic Object Tracking

In today's tutorial, I will briefly go over how to use GAAS-Object-Tracking module to track an object with UAVs.

Previous无人机自动驾驶软件系列 E05: 视觉引导降落Next无人机自动驾驶软件系列 E06: 简单全局目标追踪

Last updated 5 years ago

Was this helpful?

Give us a Star on if you find this tutorial useful.

Help us to understand what you want to learn in the upcoming tutorials by fill out this .

Join our

---------------------

We suggest developers to install GAAS mirror to use the project:

In the previous tutorials, we talked about how to set up a simulation, how to use Python to control UAVs and how to use stereo cameras to estimate depth. In today's tutorial, I will briefly go over how to use module to track an object with UAVs. Please note, this realization is very basic. If you would like to use it in a real operating environment, you may need to use a more complex realization.

In order to better maintain the repo, we have uploaded the tracking algorithms . You may use the following command to clone the tracking module to your local environment:

git clone https://github.com/generalized-intelligence/GAAS-Object-Tracking.git

GAAS Tracking module consists of four algorithms, which are , , , and based SiamRPN. The former two algorithms are traditional computer vision algorithms and the latter two are based on neural networks. In this tutorial, KCF is our algorithm of choice.

Please note that GAAS-Object-Tracking is a module of the GAAS project. It doesn't work as a stand-alone tracking software.

Update Environment

First, update GAAS:

git pull origin master

Next, copy all files required by Gazebo simulations to the directory of PX4:

cp -r (GAAS_PATH)/simulator/launch/* ~/catkin_ws/src/Firmware/launch/
cp -r (GAAS_PATH)/simulator/models/* ~/catkin_ws/src/Firmware/Tools/sitl_gazebo/models/
cp -r (GAAS_PATH)/simulator/worlds/* ~/catkin_ws/src/Firmware/Tools/sitl_gazebo/worlds/
cp -r (GAAS_PATH)/simulator/posix-config/* ~/catkin_ws/src/Firmware/posix-configs/SITL/init/ekf2/
cp -r (GAAS_PATH)/simulator/urdf ~/catkin_ws/src/Firmware/Tools/sitl_gazebo/

Launch the Simulation

We would like to use a UAV to track a moving ground vehicle in Gazebo Simulation.

First, we need to initialize the simulation:

roslaunch px4 car_tracking.launch

The above command will initialize the simulation. In the simulation, there is a moveable ground vehicle and a UAV. The ground vehicle is your tracking target. The ground vehicle has subscribed to /cmd_vel and thus we can use either code (which I will conveniently skip for now) or a plug-in to control the ground vehicle. The plug-in can be installed by:

sudo apt install ros-kinetic-teleop-twist-keyboard

After installing the plug-in, launch the plug-in.

rosrun teleop_twist_keyboard teleop_twist_keyboard.py

After launching the plug-in, a user manual will be printed automatically.

You may press corresponding keys to control the ground vehicle.

Build the Tracking Module

We are using KCF algorithms for this tutorial with a realization in C++, so we need to build:

cd (GAAS-Object-Tracking_PATH)/KCF
mkdir build
cd build
cmake ..
make -j6

If you have encountered "fatal error: ros_kcf/InitRect.h: No such file or directory...", increase the number of threads used for building. For example, change make -j1 to make -j6

Testing

1. Initialize the Simulation Environment

First, launch the simulation environment:

roslaunch px4 car_tracking.launch

Let the UAV take off and hover at 3m of height:

cd (GAAS_PATH)/demo/tutorial_6/6_object_tracking/
python python px4_mavros_run.py

Open another terminal and control the UAV to fly to right above the ground vehicle:

python init_drone.py

Open rviz to make sure the ground vehicle is within the visual line of sight of the camera.

2. Run the Tracking Algorithm

Open a new terminal, run the tracking algorithm with:

cd (GAAS-Object-Tracking_PATH)/KCF
./bin/startRosService

Note: There is no output at this step. The algorithm is waiting for the initial respective coordinates of the ground vehicle (in the camera frame) from ROS Service.

GAAS provides a GUI-based software to quickly set the initial coordinates. Use the following command to launch the software:

cd (GAAS_PATH)/demo/tutorial_6/6_object_tracking/
python set_init.py

In order to keep the environmental variables lean, let's manually modify set_init.py to add all dependencies for ROS Service to the list of environmental variables.

First, use an editor to open set_init.py

gedit (GAAS_PATH)/demo/tutorial_6/6_object_tracking/set_init.py

You should be looking at the following variable:

import rospy
import cv2
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
import numpy as np 

import sys 
sys.path.append('(GAAS-Object-Tracking_PATH)/KCF/build/devel/lib/python2.7/dist-packages')
from ros_kcf.srv import InitRect    #Import this ROS Service dependency
from std_msgs.msg import Int32MultiArray

Save the change and close the editor. Run set_init.py. In the pop up window, draw a rectangle to enclose the ground vehicle and press "s" with the keyboard to send the initial coordinates to the tracking algorithm through ROS Service.

After you have completed the above steps, you should see outputs in the terminal where you ran the tracking algorithm. Now you may press ctrl + c to close the program.

The tracking algorithm by default subscribes /gi/simulation/left/image_raw topic. If you would like to change it, you may change the config file:

gedit (GAAS-Object-Tracking_PATH)/KCF/config.yaml

The GUI-based software is very basic. We will continue working on upgrading the software. Any contributions will be well appreciated along the way.

3. Start Tracking

First, let's set the parameters. You may set the UAV to fly at a certain height, or you may choose to keep a fixed distance between the UAV and the ground vehicle based on depth estimation with stereo cameras. The software supports StereoBM to tune the parameters. To keep things simple, in this particular tutorial, we will set the UAV to fly at a certain height. If you would like to let the UAV fly at a different height, change the corresponding parameter.

Launch the algorithm:

python track_and_move.py

Launch the ground vehicle controller plug-in:

rosrun teleop_twist_keyboard teleop_twist_keyboard.py

Control the ground vehicle as you wish.

During Tracking, do not stop the tracking algorithm. If you would like to start over, re-launch the tracking algorithms and run set_init.py to reset the initial coordinates.

We will continue working on improving the tracking and depth estimation. This tutorial aims to provide a basic realization of ground vehicle tracking.

The ground vehicle in this tutorial is extremely basic. If you would like to build a more advanced ground vehicle, you may use module. The control is also based on /cmd_vel .

You may also revisit to build your own command file to fly the UAV to above the ground vehicle.

In this tutorial, both the UAV and the ground vehicle are facing the positive direction in the X-axis by default. If you would like to change the default facing direction, please modify the code. If you would like to use a global frame of reference, please see codes from .

Give us a Star on if you find this tutorial useful.

Help us to better understand what you want to learn in the upcoming tutorials by fill out this .

🌟
Github
three-question survey
slack
GAAS v0.7 Release Mirror - x64
GAAS-Object-Tracking
here
TLD
KCF
GOTURN
pysot
husky_gazebo
Autonomous Drone Software E01
Autonomous Drone Software E01
🌟
Github
Support us on Patreon.
three-question survey
UAV and the ground vehicle in a simulated environment
teleop_twist user manual
UAV hovering on top of the ground vehicle
View from the UAV's perspective
Set initial coordinates
Real-time result of object tracking
The UAV is following the ground vehicle