Segmentation camera

Simulated ANAFI Ai embeds two segmentation cameras, for research and experiment purposes only. One is located at the same position as the front camera, the other is at the same position as the right stereo camera. They have the same camera characteristics as the corresponding “real” camera at the same location (same resolution and framerate).

By default, every object in the scene is assigned to a stencil value in the range 0-255 which is then mapped to a specific RGB color from a predefined color palette. The following stencil values are reserved to specific objects in the scene:

Value	Alias	Converted color	Description
0	None	(0, 0, 0)	Objects that have not been categorized are considered Unlabeled. By default, this category is empty.
1	Drones	(153, 108, 6)	Drones spawned by Parrot Sphinx.
2	Characters	(112, 105, 191)	Humans that walk or ride/drive any kind of vehicle or mobility system.
3	Vehicles	(89, 121, 72)	Cars, vans, trucks, motorcycles, bikes, buses, trains.
4	Sky	(190, 225, 64)	Sky atmosphere (clouds, stars, sun, …).

Example 1: Stereo camera segmentation

Example 2: Front camera segmentation

Record and visualize segmentation images

RGB images produced by segmentation cameras can be visualized and recorded just like images produced by other types of cameras with Parrot Sphinx tools (see Record and visualize cameras for more information).

Set stencil values for a specific level

The easiest way to set your own stencil values is by modifying a template stencil file generated by the UE4 application itself.

To generate a template stencil file, use the command line option -write-stencil-file=<file>:

$ parrot-ue4-empty -write-stencil-file=/path/to/template.yaml

The generated file contains:

• The default option settings

• The list of reserved aliases

• Some randomly picked-up objects that have been assigned a value

• [comments] The whole list of paths to the existing meshes

• [comments] The whole list of object instance IDs

• [comments] The color palette mapping stencil values and RGB colors

After being edited, the file can be passed as command line argument to the UE4 application with the option -stencil-file=<file>:

$ parrot-ue4-empty -stencil-file=/path/to/template.yaml

A stencil file uses the YAML format to associate stencil values with objects in the level.

First of all, the file set some options defining how the segmentation cameras should work:

• AssignRandomStencilValues: boolean parameter indicating whether random stencil values are assigned to all objects by default.

• OverrideExisting: boolean parameter indicating whether preexisting stencil values will be preserved or overridden by default values (either random or zero).

• 3DGridEnabled and 3DGridBlockSize: setting 3DGridEnabled to true activates an advanced mode mainly thought to evaluate 3D scanning performed by the simulated drone. This mode is described in a dedicated section: Recipe: 3D-grid segmentation to evaluate 3D scanning.

It may also contain an Aliases that binds alias names with stencil values. Note that some aliases are predefined as listed at the beginning of this section.

It must contain an Objects array where each item is a map where:

• Name is a regular expression respecting the ECMAScript standard and used to find the corresponding object(s)

• either Value is a stencil value,

• or Alias is one the aliases defined in the Aliases map.

An object can be either a path to a generic mesh, or a specific object instance defined by its ID.

Example of a stencil file:

AssignRandomStencilValues: true
OverrideExisting: false
Aliases:
    None: 0
    Drones: 1
    Characters: 2
    Vehicles: 3
    Sky: 4
    Rocks: 5
Objects:
  - Name: /Engine/EngineSky/SM_SkySphere.SM_SkySphere
    Alias: Sky
  - Name: /Game/Forest/Meshes/Rocks/.*
    Alias: Rocks
  - Name: /Game/Forest/Meshes/Trees/.*DeadTrunk.*
    Value: 6
  - Name: /Game/Forest/Meshes/Trees/.*FallenTree.*
    Value: 7