Mission UI¶

Mission UI allows you to add User Interface (called UI) items in the piloting application FreeFlight communicating with your mission running on the drone. This allows for example to have mission settings available for the pilot, controls and mission-specific information in the piloting screen.

Important

Mission UI is only available for ANAFI UKR missions, using the FreeFlight8 application or higher versions. This feature is only available with Python-based services.

This documentation provides the foundation to get started with Mission UI. For complete examples, refer to the Air SDK samples.

Architecture overview¶

Mission UI is manifested in FreeFlight by webviews, and therefore uses web-related technologies: HTML, CSS, JS and images. They all are later known as UI items. However, the mission installed and running on the drone is still coded in Python.

The UI items are embedded as assets in the mission and made available for download by the drone’s webserver. The FreeFlight application downloads the assets when needed to display the UI webview.

Communication between the mission on the drone and the mission UI in FreeFlight is made possible using websockets. The messages contents are up to the mission developer.

This communication is managed by the drone, transmitting data through the drone<->remote radio link and the remote<->tablet USB-C/Lightning link.

../_images/mission_ui_detailed_interactions.png

Embedding assets¶

Asset files¶

Mission UI asset files are HTML files and associated resources (JS, CSS, images…). When present in the mission source tree these files are embedded in the mission archive once built. Therefore they are installed on the drone along with the mission.

Any file can be embedded as an asset in a mission archive, but only specific HTML files will be used by the FreeFlight application as mission UI items (see mission UI layouts below).

Asset files embedded in a mission installed on the drone are made available by the drone’s webserver for the application to download.

The Mission UI file structure is fixed and must not be altered. Files should remain in their designated locations to ensure proper integration with the mission system.

The Mission UI file structure is as follows:

<projectFolderName>
├── assets
│   └── static
│       ├── ff8-action-bar.html
│       ├── ff8-settings.html
│       ├── ff8-side-panel.html
│       ├── ff8-icon.png
│       ├── ff8.css
│       ├── socket-helpers.js
│       └── script.js
├── services
│   └── <service_name>
│       ├── __init__.py
│       ├── appserver.py
│       ├── main.py
│       └── uid.py
└── mission.yaml

ff8-action-bar.html is the action bar layout (see ff8-action-bar.html for Mission UI example).
ff8-settings.html is the settings layout (see ff8-settings.html for Mission UI example).
ff8-side-panel.html is the side panel layout (see ff8-side-panel.html for Mission UI example).
ff8-icon.png is the logo displayed onto the remote SkyController. This logo will be the same between the main view and the settings view. Its size must be 72x72 pixels, be black on a transparent background. The default icon is ff8-icon.png
ff8.css is the stylesheet. You can override it or use yours (see ff8.css).
socket-helpers.js is the file which handles the WebSocketHelper implementation (see socket-helpers.js).
script.js is your custom JavaScript file, responsible for sending commands and handling responses.
appserver.py implements a websocket server using FastAPI to manage connections and messages.
main.py starts this server and registers the accepted commands.
uid.py defines the unique identifier of the mission.
mission.yaml is the top level description of the mission, with associated dependencies (see mission.yaml file format).

During the airsdk-cli init, the following line will appear if the selected service language is Python:

Mission UI allows users to have remote UI through a webview. A layout is
available for three sections: Settings, Side panel and Action bar. This
function is only available in Python and is not compatible with ANAFI Ai.
Note: For more details refer to https://developer.parrot.com/docs/airsdk/general/mission_ui.html

[?] Are Mission UI layouts needed ?: yes
> yes
no

Choosing yes (default) will automatically set-up the working filetree for Mission UI, generate the assets folder and others files such as: appserver.py, uid.py, and will replace the main.py (which includes an operational template for Mission UI).

Graphic design¶

To ensure that the mission UI design integrates smoothly with the FreeFlight application, a ff8.css file is provided defining the common widget styles. This file is automatically added to the project skeleton when creating a new mission with airsdk-cli.

The following image summarizes all available widgets and styles. It’s a capture of the rendering of the mission_ui_samples.html file inlined below.

You can find more information about the main style guidelines in this annex: Mission UI HTML+CSS Guidelines

Mission UI layouts¶

Three Mission UI layouts are available in FreeFlight:

Settings: Displayed in the mission configuration screen
Side Panel: A side panel displayed in flight main screen
Action Bar: Action buttons available in flight main screen

Each HTML file in assets/static corresponds to one of these layouts. If a file is present, the corresponding layout will be displayed.

Important

You cannot use the side panel and the action bar simultaneously. If both are set, only the side panel will be displayed by FreeFlight.

To maintain consistency with FreeFlight, the provided ff8.css file defines standard UI styles and components (see ff8.css) Developers can extend or override these styles by including an additional CSS file, giving them the flexibility to choose whether to modify ff8.css or create their own custom styles.

Settings¶

Settings are implemented in the settings.html asset.

Side panel¶

The side panel is implemented in the side-panel.html asset.

Action bar¶

The action bar is implemented in the action-bar.html asset.

Communication between the mission and mission UI¶

The communication between the mission running on the drone and the UI is handled via a webserver and websockets. The webserver is built with FastAPI (ideally version 0.115.12 for optimal compatibility and stability) and runs on Uvicorn providing an efficient and asynchronous environment for real-time interactions.

Each HTML view opens a websocket connection to enable real-time communication. On the drone side, all necessary actions should be defined in the Python service files, while the UI components (HTML, CSS, and JavaScript) are placed in the assets/static directory. Actions triggered by the user through UI buttons or other interactions send websocket commands to the drone, which processes them accordingly.

The mission developer must declare a service on the drone via a dedicated API. This propagates the service IP address and port to the webview, allowing the websocket client to connect.

WebSocket Server (appserver.py)¶

The webserver acts as the communication bridge between the tablet (running FreeFlight with Mission UI) and the drone. Since the tablet and SkyController do not store any data, once the drone is turned off, all UI items are lost and will be automatically reloaded upon reconnection. To prevent data loss, the drone’s server could store the mission context. Alternatively, you can use the localStorage on the tablet to store mission-related data or UI state. This enables the interface to be quickly and partially restored upon reconnection, minimizing disruptions and ensuring a seamless user experience.

The appserver.py file:

Manages websocket connections (acceptance, disconnection, sending, and broadcasting messages).
Starts a FastAPI-based websocket server using Uvicorn.
Provides a WebSocketServer class that supports dynamic command registration.
Retrieves the server port dynamically and publishes it via Air SDK.
Parses incoming messages and executes the corresponding command.
Sends responses back to the connected clients.

Command Handling (main.py)¶

This file captures incoming data, processes it, and sends responses back to the tablet or controller, ensuring seamless interaction.

The main.py file:

Initializes the WebSocketServer instance.
Registers all available commands.
Implements business logic for processing incoming and outgoing messages.
Publishes the websocket server port via airsdkd publisher.
Sends processed data back to the UI when required.

WebSocket Client (socket-helpers.js and script.js)¶

To facilitate websocket communication, the WebSocketHelper JavaScript class is implemented (socket-helpers.js). This file establishes a websocket connection with the drone’s server for each webview and make possible the interaction between the UI and the drone mission. It provides easy-to-use methods for handling websocket events.

WebSocketHelper provides the following accessible methods:

onSocketOpened(event) - Triggered when the websocket connection is established.
onSocketClosed(event) - Triggered when the connection is closed.
onSocketError(event) - Triggered in case of a websocket error.
sendMessage(message, name) - Sends commands from the Mission UI to the drone.
onMessageReceived(event) - Receives and handles responses from the drone.

The custom JavaScript file (script.js) handles both sending and receiving messages on the UI/tablet side. It processes incoming messages to update the user interface and allows users to send messages via input fields and buttons, for example.

Communication Overview Examples¶

The first flow demonstrates the user interaction required to receive a response from the drone. Its flow follows this structure:

The UI triggers an event (e.g., button press).
A websocket message is sent to the drone.
The websocket server processes the request and executes the corresponding command.
The drone responds with relevant data.
The UI updates accordingly based on the received response.

Example of JSON communication, when a message is sent from the UI:

script.js¶

{
    function sendMsg(msg) {
        console.log('Send: ' + JSON.stringify(msg));
        WebSocketHelper.sendMessage(JSON.stringify(msg));
    }
}

message sent¶

{
    "cmd": "sendMsg",
    "value": "Hello, Drone!"
}

The websocket server processes the message and responds:

script.js¶

 WebSocketHelper.onMessageReceived = (event) => onRecvMsg(event.data);

 function onRecvMsg(data) {
     var msg = JSON.parse(data);
     if (msg.hasOwnProperty("sendMsg")) {
         msgValue = msg.sendMsg;
         /* Some processing here */
     }
 }

message received¶

 {
     "sendMsg": "Drone received the message Hello, Drone!"
 }

A second communication flow is represented in the Mission UI samples, which transmits some IMU Telemetry (Gyroscope and Accelerometer data) to the side- panel webview. This flow involves data being transmitted without any user interaction.

Communication flow synthesis (simplified)¶

Mission UI and SkyController Access¶

Mission UI services running on the drone are exposed as AirSDK mission services. By default, these services are only available inside the drone internal network. To allow external applications (e.g., a laptop connected to the SkyController) to access the Mission UI WebSocket endpoints, the mDNS Republisher (republisherd) automatically republishes mission services on the SkyController network.

This means:

The Mission UI WebSocket server (appserver.py) can be discovered and connected to via the SkyController, without any special network setup.
Users can interact with Mission UI services for development, debugging, or control purposes from any device on the SkyController network.

For detailed behavior of the republisher and how it exposes AirSDK mission services, see mDNS Republisher.

Debugging with SkyController¶

When connected to the SkyController (e.g. via Ethernet), you can access the drone’s webserver through a proxy on port 180. The base address is http://192.168.53.1:180.

To access a Mission UI webview, use the following URL format:

http://192.168.53.1:180/data/missions/<mission_uid>/<webview_name>.html?<service_name>_address=192.168.53.1&<service_name>_port=<republished_port>

Note that:

The IP address is the SkyController’s IP (192.168.53.1).
The port 180 is used to access the drone webserver and the missions HTML webviews which are hosted on it.
The <service_name>_port must be the port assigned by the mDNS republisher (see mDNS Republisher (republisherd)) for your service on the SkyController. You can use Avahi to retrieve it.

Debugging with USB¶

If you are connected directly to the drone via USB, you can access the webviews by knowing the service port. You can find the port in the drone mission logs or retrieve it from your computer using Avahi.

Example URL:

http://192.168.43.1/data/missions/<mission_uid>/<webview_name>.html?<service_name>_address=192.168.43.1&<service_name>_port=xxxx

To use this URL, replace <mission_uid> with your Mission UID, <service_name> with the name of the service you want to connect to (e.g., my_service), and replace xxxx with the actual port number associated with the service. For example, if the mission UID is com.parrot.missions.samples.mission_ui, the webview you want to see is ff8-side-panel, your service is called my_service and the port number is 45454, the URL would look like this: http://192.168.43.1/data/missions/com.parrot.missions.samples.mission_ui/ff8-side-panel.html?my_service_address=192.168.43.1&my_service_port=45454

This allows testing the UI without needing a physical controller.

You can also use the drone’s URL instead of its IP address : http://anafi-ukr.local

To debug websocket communication, you can:

Use the web browser developer console (F12 -> Console/Network tabs) to inspect websocket messages.
Send test websocket messages using tools like wscat or Postman.
Monitor Drone mission logs to track incoming/outgoing messages.

For more code samples, refer to the Mission UI annexes.