Autonomous Drone Software E06: Basic Object Tracking
In today's tutorial, I will briefly go over how to use GAAS-Object-Tracking module to track an object with UAVs.
Last updated
In today's tutorial, I will briefly go over how to use GAAS-Object-Tracking module to track an object with UAVs.
Last updated
Give us a Star on Github if you find this tutorial useful.
Help us to understand what you want to learn in the upcoming tutorials by fill out this three-question survey.
Join our slack
---------------------
We suggest developers to install GAAS mirror to use the project:
In the previous tutorials, we talked about how to set up a simulation, how to use Python to control UAVs and how to use stereo cameras to estimate depth. In today's tutorial, I will briefly go over how to use GAAS-Object-Tracking module to track an object with UAVs. Please note, this realization is very basic. If you would like to use it in a real operating environment, you may need to use a more complex realization.
In order to better maintain the repo, we have uploaded the tracking algorithms here. You may use the following command to clone the tracking module to your local environment:
GAAS Tracking module consists of four algorithms, which are TLD, KCF, GOTURN, and pysot based SiamRPN. The former two algorithms are traditional computer vision algorithms and the latter two are based on neural networks. In this tutorial, KCF is our algorithm of choice.
Please note that GAAS-Object-Tracking is a module of the GAAS project. It doesn't work as a stand-alone tracking software.
First, update GAAS:
Next, copy all files required by Gazebo simulations to the directory of PX4:
We would like to use a UAV to track a moving ground vehicle in Gazebo Simulation.
First, we need to initialize the simulation:
The above command will initialize the simulation. In the simulation, there is a moveable ground vehicle and a UAV. The ground vehicle is your tracking target. The ground vehicle has subscribed to /cmd_vel and thus we can use either code (which I will conveniently skip for now) or a plug-in to control the ground vehicle. The plug-in can be installed by:
After installing the plug-in, launch the plug-in.
After launching the plug-in, a user manual will be printed automatically.
You may press corresponding keys to control the ground vehicle.
The ground vehicle in this tutorial is extremely basic. If you would like to build a more advanced ground vehicle, you may use husky_gazebo module. The control is also based on /cmd_vel .
We are using KCF algorithms for this tutorial with a realization in C++, so we need to build:
If you have encountered "fatal error: ros_kcf/InitRect.h: No such file or directory...", increase the number of threads used for building. For example, change make -j1 to make -j6
First, launch the simulation environment:
Let the UAV take off and hover at 3m of height:
Open another terminal and control the UAV to fly to right above the ground vehicle:
You may also revisit Autonomous Drone Software E01 to build your own command file to fly the UAV to above the ground vehicle.
Open rviz to make sure the ground vehicle is within the visual line of sight of the camera.
Open a new terminal, run the tracking algorithm with:
Note: There is no output at this step. The algorithm is waiting for the initial respective coordinates of the ground vehicle (in the camera frame) from ROS Service.
GAAS provides a GUI-based software to quickly set the initial coordinates. Use the following command to launch the software:
In order to keep the environmental variables lean, let's manually modify set_init.py to add all dependencies for ROS Service to the list of environmental variables.
First, use an editor to open set_init.py
You should be looking at the following variable：
Save the change and close the editor. Run set_init.py. In the pop up window, draw a rectangle to enclose the ground vehicle and press "s" with the keyboard to send the initial coordinates to the tracking algorithm through ROS Service.
After you have completed the above steps, you should see outputs in the terminal where you ran the tracking algorithm. Now you may press ctrl + c to close the program.
The tracking algorithm by default subscribes /gi/simulation/left/image_raw topic. If you would like to change it, you may change the config file:
The GUI-based software is very basic. We will continue working on upgrading the software. Any contributions will be well appreciated along the way.
First, let's set the parameters. You may set the UAV to fly at a certain height, or you may choose to keep a fixed distance between the UAV and the ground vehicle based on depth estimation with stereo cameras. The software supports StereoBM to tune the parameters. To keep things simple, in this particular tutorial, we will set the UAV to fly at a certain height. If you would like to let the UAV fly at a different height, change the corresponding parameter.
Launch the algorithm:
Launch the ground vehicle controller plug-in:
Control the ground vehicle as you wish.
During Tracking, do not stop the tracking algorithm. If you would like to start over, re-launch the tracking algorithms and run
set_init.pyto reset the initial coordinates.
In this tutorial, both the UAV and the ground vehicle are facing the positive direction in the X-axis by default. If you would like to change the default facing direction, please modify the code. If you would like to use a global frame of reference, please see codes from Autonomous Drone Software E01.
We will continue working on improving the tracking and depth estimation. This tutorial aims to provide a basic realization of ground vehicle tracking.
Help us to better understand what you want to learn in the upcoming tutorials by fill out this three-question survey.
Give us a Star on Github if you find this tutorial useful.
