Road Runner mission¶

The Road Runner mission is a more complete mission sample than the previous examples. This mission uses the front camera and computer vision algorithm to follow a line on a road. The Road Runner mission involves:

a custom Flight supervisor with custom states
a few Guidance modes (overridden and custom modes) in Python and C++
a service to access the Video and Telemetry APIs and compute the trajectory of the drone

Note

Actually, the Road Runner mission follows a line in the the center of the road and not the road itself.

Important

This mission was only tested in a simulated environment using Sphinx. Due to the simple computer vision algorithm used in this mission, it could be troublesome to run this mission on a real drone. The computer vision is very sensitive to the light variation, the color of the line should be easily distinguishable from the rest to run this mission without problems.

The source code is available here.

Getting started¶

Build, install and run the Road Runner sample mission¶

Please follow the instruction of the README file.

Mission overview¶

Road Runner sample mission file tree:

road_runner
    ├ assets -------------------- Files to be directly copied in the mission installable archive
    ├ fsup ---------------------- Flight supervisor directory
    |  ├ flying------------------ Flying stage
    |  |  └ road_following ------ Custom state: road_following + look_down
    |  ├ takeoff----------------- Takeoff stage
    |  |  └ road_runner --------- Overridden state: ascent
    |  ├ mission.py ------------- Main mission file: contains the state machine
    |  └ uid.py ----------------- UID of the mission
    ├ guidance
    |   ├ look_down ------------- Custom guidance mode: Python
    |   └ road_following -------- Custom guidance mode: C++
    ├ msghub
    |   ├ cv_road --------------- Service messages
    |   └ guidance -------------- Guidance messages
    ├ services
    |   └ cv_road
    |      ├ src
    |      |  ├ listener
    |      |  ├ processing ------ Main processing
    |      |  └ video
    |      └ main.cpp ----------- Service main file which call processing object
    └ mission.yaml -------------- Mission configuration file

How does it works¶

../../_images/road_runner_overview_dark.png

Fsup State machine¶

After receiving a takeoff command, the drone enters the overriden takeoff state from the custom road runner mission state machine. The Road Runner takeoff state modifies the ascent target to 10 meters. Once the target is reached, the next state is the set_gimbal state. This state sends commands to point the front camera to the ground. Finally, the terminal state is the road_following state which call the road_following guidance mode.

Fsup makes msghub bridge between the road_following guidance mode mode and the cv_road service.

When Fsup receives a move command, or the message road_lost from the cv_road service, the state machine makes the transition to the default flying.hoveride state.

When Fsup receives a takeoff/landing command, the state machine makes the transition to the default landing state.

Guidance¶

The state takeoff is the same as the default takeoff state. The only
difference is the target height of 10 metres.
The state set_gimbal is a python guidance mode which
sets the pitch front camera position and sends a message done to fsup once the target is reached.
The state road_following is a C++ guidance mode which
generates the trajectory of the drone with the telemetry values received from the cv_road service. It sends a message road_followin_enable or road_followin_disable to activate or not the video processing of the cv_road service.

Service¶

The cv_road service processes the video of the front camera with OpenCV to extract the position of the line in the center of the road. The line mean direction can be detected with a simple HSV color mask. A canny filter is applied to detect the edges of the line. Then, a Hough Line Transform is applied to detect straight lines. Finally the road line mean direction is obtained by applying the average of all the detected lines.

../../_images/road_runner_original_frame.png — Original frame¶

../../_images/road_runner_mask_frame.png — HSV color mask¶

../../_images/road_runner_canny_frame.png — Canny filter¶

../../_images/road_runner_final_frame.png — Final Frame. The blue lines are all the lines detected by the *Hough Line Transform* function and the red line is the mean of all theses lines¶

Modify color filter¶

To run this mission sample in an other simulated world or on a physical drone, the HSV color mask needs to be modified to detect another line.

/* processing.cpp */

static void do_step(const struct vipc_frame *frame,
            struct roadData *road_data,
            bool &is_road_detected)
{
    /* [...] */

    cv::inRange(input_frame, cv::Scalar(low_H, low_S, low_V), cv::Scalar(high_H, high_S, high_V) output_frame);

    /* [...] */
}

When the line is not detected for 5 seconds, the service sends the message road_lost to fsup and the drone returns to hover mode.