Camera

Simulated cameras are highly tunable. Many parameters are available through the Web dashboard and sphinx-cli. Given their large quantity, they have been organized in a hierarchical way.

  • general settings: At the root level of the camera parameters, we can find parameters covering essential characteristics of the camera such as frame rate, field of view, delay, etc.

  • clip: Modify near and far clips to control field of rendering.

  • offsets: Add deltas to the original position and orientation of the camera.

  • post_process/auto-exposure: automatic light adjustment of the current scene viewed by the camera.

  • post_process/camera: Settings of the camera lens. Can impact the depth of field effect and the exposure if set in ‘manual’ mode.

  • post_process/bloom effect: blur caused by bright lights

  • post_process/chromatic aberrations: simulates the color shifts in real-world camera lenses.

  • post_process/color grading: alter or enhance the overall color of light in the scene.

  • post_process/motion blur: Generates a Motion Blur effect that blurs objects based on its motion.

  • post_process/tonemapper: map the wide range of high dynamic range (HDR) colors into low dynamic range (LDR) that a display can output.

  • post_process/vignette: simulates the darkening of the borders of real-world camera lenses.

  • post_process/lens flares: Simulates the scattering of light when viewing bright objects through imperfections found in camera lenses:

    • corona_rays1 and corona_rays2: configure the two sets of rays that simulate the Sun’s corona.

    • ghost1, ghost2, ghost3, ghost4: configure the ghosting effects that occur when light repeatedly reflects off the surface of the lens and is seen in the image.

    • halo: configures the main halo around the Sun with chromatic aberration.

    • sun_disk: configures the Sun disk and intensity.

    • sun_rays: configures the rays of light caused by atmospheric diffraction.

  • noise: Three types of noise can be exclusively applied.

    • shot_noise. Realistic camera noise.

    • gaussian. Spatial gaussian noise.

  • burnt_sky: simulates photodiode saturation effect of the camera sensor.

  • distortion: simulates the distortion effect produced by the camera lens.

  • wide_angle_lens: simulates a wide-angle camera sensor. For moderate view angles, lens distortion can be simulated with distortion parameters. But, if you need a camera with fisheye, or other types of wide-angle lens whose field of view is larger than 120°, then you should use wide_angle_lens parameters. Note that when distortion is enabled, wide_angle_lens parameters are ignored. The parameter mapping_function can take the following strings:

    • equidistant: R = f * DistortedTheta

    • stereographic: R = 2 * f * tan(DistortedTheta / 2)

    • equisolid_angle: R = 2 * f * sin(DistortedTheta / 2)

    • orthographic: R = f * sin(DistortedTheta)

    • kumler_bauer: R = k1 * sin(k2 * DistortedTheta). This function takes two coefficients in parameter coeffs. The first one is “f / k1”. The second one is “k2”.

    ..where :

    • “R” is the radial distance from the sensor center to the image point

    • “DistortedTheta” is the angle formed by the ray of light and the normal to the sensor plane.

    • “f” is the focal length of the sensor

  • gui: Various settings having an effect on the HMI.

  • depth/disparity: Configuration of special cameras dedicated to compute depths or disparities.

Note

The “post_process” parameters described above come from the “post_process” step natively offered by UE. They have been categorized the same way as in UE. More details can be found HERE).

All parameters

Name

Description

Expected type

Read-only

framerate

Time in seconds between two consecutive captures. If set to zero, the camera captures only upon asynchronous triggers, like instantaneous_start and sync_over modes

floating-point number

N

horizontal_fov

Horizontal field of view in radians.

floating-point number

N

image_too_dark_expr

Expression setting the too dark alert in the firmware. In addition to predefined variables, it has access to the following variable: luminance (Last frame luminance value in EV100), It has no effect in manual auto exposure mode. Its default value is: luminance + inside_area('indoor') < 1.0

string

N

out_of_order

If true, set the camera out of order. No frame is sent anymore.

boolean

N

burnt_sky/

Name

Description

Expected type

Read-only

debug

Internal use. Renders a gradient red color when pixels are burnt.

boolean

N

enabled

Flag enabling the burnt sky effect: When sky luminance is high enough, parts of objects in foreground may be masked due to photodiodes saturation.

boolean

N

luminance_thr

Beyond this luminance value [0. 1.], the burnt sky effect is activated. It is advised to set a value close to 1.

floating-point number

N

saturation_influence

The lower the coefficient [0. 1.] is, the more saturated the pixel needs to be to get a large saturation area around the pixel.

floating-point number

N

clip/

Name

Description

Expected type

Read-only

far

Camera’s far clip distance in meters

floating-point number

N

near

Camera’s near clip distance in meters

floating-point number

N

depth/

Name

Description

Expected type

Read-only

baseline

Baseline of the stereo camera in meters. Set this value to 0 to disable the quantization effect and render the true depth.

floating-point number

N

drone_mask_color

Pixel value indicating that the depth is masked by the drone body.

floating-point number

N

drone_mask_enabled

If true, enable drone masking.

boolean

N

gaussian_noise/mean

Mean of the gaussian noise distribution. [0. 1.]

floating-point number

N

gaussian_noise/stddev

Standard deviation of the gaussian noise distribution. Noise is applied on floating number colors belonging to [0. 1.].

floating-point number

N

inv_px_color/[r g b a]

Reference color to invalidate pixels. Deactivated if the values provided are negative at least for RGB.

floating-point number

N

inv_px_rate

Rate of invalid pixels taken randomly in the depth map. 0. means no invalid pixels using this method.

floating-point number

N

max_disparity

Highest disparity value before saturating.

floating-point number

N

native_cam_width

Stereo camera width in pixels.

integer

N

use_native_width

If true, use native camera’s resolution to capture depths.

boolean

N

disparity/

Name

Description

Expected type

Read-only

baseline

Baseline of the stereo camera in meters.

floating-point number

N

drone_mask_color

Pixel value indicating that the depth is masked by the drone body.

integer

N

drone_mask_enabled

If true, enable drone masking.

boolean

N

gaussian_noise/mean

Mean of the gaussian noise distribution. [0. 1.]

floating-point number

N

gaussian_noise/stddev

Standard deviation of the gaussian noise distribution. Noise is applied on floating number colors belonging to [0. 1.].

floating-point number

N

inv_px_color/[r g b a]

Reference color to invalidate pixels. Deactivated if the values provided are negative at least for RGB.

floating-point number

N

inv_px_rate

Rate of invalid pixels taken randomly in the disparity map. 0. means no invalid pixels using this method.

floating-point number

N

max_disparity

Highest disparity value before saturating.

floating-point number

N

native_cam_width

Stereo camera width in pixels.

integer

N

distance/

Name

Description

Expected type

Read-only

drone_mask_color

Pixel value indicating that the distance is masked by the drone body.

floating-point number

N

drone_mask_enabled

If true, enable drone masking.

boolean

N

distortion/

Name

Description

Expected type

Read-only

center/[x y]

Distortion center or principal point expressed as ratios in X and Y dimensions [0.0, 1.0].

floating-point number

N

crop

If true, enable cropping.

boolean

N

debug

Internal use. Renders the test pattern.

boolean

N

enabled

If true, enables distortion effect

boolean

N

k1

Radial distortion coefficient k1

floating-point number

N

k2

Radial distortion coefficient k2

floating-point number

N

k3

Radial distortion coefficient k3

floating-point number

N

p1

Tangential distortion coefficient p1

floating-point number

N

p2

Tangential distortion coefficient p2

floating-point number

N

gui/

Name

Description

Expected type

Read-only

color/[r g b a]

Color of the visualization artifacts and of the frame panel associated with the camera.

floating-point number

N

visualize

If true, display in the main HMI window the orientation and FoV of the camera.

boolean

N

visualize_hdr

If true, display on the generated frame the current auto-exposition histogram.

boolean

N

noise/

Name

Description

Expected type

Read-only

debug

Internal use. Works only when a noise kind is selected. Generate a uniform gray texture.

boolean

N

debug_saturations

Internal use. Renders black pixels in magenta, and white pixels in green.

boolean

N

gaussian_noise/mean

Mean of the gaussian noise distribution. [0. 1.]

floating-point number

N

gaussian_noise/stddev

Standard deviation of the gaussian noise distribution. Noise is applied on floating number colors belonging to [0. 1.].

floating-point number

N

noise_type

Name of the noise type to be applied.

[_, gaussian, shot]

N

shot_noise/debug

Internal use. Renders original color if noise is almost zero, red otherwise.

boolean

N

shot_noise/illum_expr

ExprTk expression of the illumination (in lux) used by the shot noise. It may depend on the variable exposure_gain, which is the resulting gain from the auto-exposure process. No space allowed in the expression.

string

N

shot_noise/illum_reference

If the scene illumination gets below the provided value, the noise due to camera’s analog gain starts being produced.

floating-point number

N

shot_noise/lens_shading/lens_radius

Maximal radius value within a frame. It has no unit, and impacts the shading_expr computation.

floating-point number

N

shot_noise/lens_shading/sensibility_factor

Lens shading effect’s contribution factor. Determines the extent of shading on pixels: a higher factor results in more shaded pixels.

floating-point number

N

shot_noise/lens_shading/shading_expr

Expression describing how the lens shading effect is distributed in the frame. r is the radius from the lens center.

string

N

shot_noise/lens_shading/values

Array of lens shading correction values separated by spaces, where their index corresponds to the distance from the optical center of the lens. These values are applied inversely to create the corresponding lens shading effect. If empty string, the shading expression is used instead.

string

N

shot_noise/s1_stddev_factor

Factor influencing the random distribution S1 of shot noise.

floating-point number

N

shot_noise/s2_stddev_factor

Factor influencing the random distribution S2 of shot noise.

floating-point number

N

shot_noise/scene_illuminance

Deprecated.

floating-point number

N

offsets/

Name

Description

Expected type

Read-only

position_x_expr

Expression setting the X offset of the camera expressed in meters in the frame of the camera.

string

N

position_y_expr

Expression setting the Y offset of the camera expressed in meters in the frame of the camera.

string

N

position_z_expr

Expression setting the Z offset of the camera expressed in meters in the frame of the camera.

string

N

rotation_pitch_expr

Expression setting the pitch offset of the camera expressed in radians in the frame of the camera.

string

N

rotation_roll_expr

Expression setting the roll offset of the camera expressed in radians in the frame of the camera.

string

N

rotation_yaw_expr

Expression setting the yaw offset of the camera expressed in radians in the frame of the camera.

string

N

optical_flow/

Name

Description

Expected type

Read-only

mask_pixels

If true, set pixel motion to zero for pixels that are out of the field of view of the previous frame.

boolean

N

post_process/

Name

Description

Expected type

Read-only

auto_exposure/compensation

Fixed logarithmic adjustment for the exposure. 0: no adjustment, -1: twice darker, -2: four times darker, 1: twice brighter, 2: four times brighter

floating-point number

N

auto_exposure/compensation_curve

Dynamic logarithmic adjustment for the exposure. The expected string is formatted as follows: X0,Y0;X1,Y1;X2,Y2;… where Xi is an average scene EV100, and Yi is the corresponding bias applied on Xi in addition to the fixed compensation. White spaces are forbidden. Interpolation is linear. If empty string, no compensation curve is applied (default).

string

N

auto_exposure/high_percent

In auto-exposure mode ‘histogram’, auto-exposure will adapt to a value extracted from the luminance histogram of the scene color. The value defines the upper percentage of the histogram that is used to find the average scene luminance.

floating-point number

N

auto_exposure/hist_log_max

In auto-exposure mode ‘histogram’, defines the upper bound for the brightness range of the generated histogram.

floating-point number

N

auto_exposure/hist_log_min

In auto-exposure mode ‘histogram’, defines the lower bound for the brightness range of the generated histogram.

floating-point number

N

auto_exposure/low_percent

In auto-exposure mode ‘histogram’, auto-exposure will adapt to a value extracted from the luminance histogram of the scene color. The value defines the lower percentage of the histogram that is used to find the average scene luminance.

floating-point number

N

auto_exposure/max_ev100

In auto-exposure modes ‘basic’ and ‘histogram’, maximum brightness for auto-exposure. The highest the eye can adapt within. Must be greater than 0 and greater than or equal to min_ev100.

floating-point number

N

auto_exposure/min_ev100

In auto-exposure mode ‘histogram’. Minimum brightness for auto exposure. The lowest the eye can adapt within. Must be less than or equal to max_ev100.

floating-point number

N

auto_exposure/mode

Auto-exposure mode. Default is histogram

[histogram, basic, manual]

N

auto_exposure/physical_exposure

Enables physical camera exposure using ShutterSpeed/ISO/Aperture.

boolean

N

auto_exposure/speed_down

In auto-exposure mode ‘histogram’, speed at which the adaptation occurs from bright to dark environment. If ‘speed_up’ or ‘speed_down’ is set to -1.0, auto-exposition gets instantaneous in both up/down directions.

floating-point number

N

auto_exposure/speed_up

In auto-exposure mode ‘histogram’, speed at which the adaptation occurs from dark to bright environment. If ‘speed_up’ or ‘speed_down’ is set to -1.0, auto-exposition gets instantaneous in both up/down directions.

floating-point number

N

bloom/intensity

Multiplier for all bloom contributions >=0: off, 0.675 (default), higher than 0.675 means brighter.

floating-point number

N

bloom/method

Name of the bloom method to be used.

[standard, convolution]

N

bloom/threshold

Minimum brightness the bloom starts having effect. -1: all pixels affect bloom equally (physically correct, faster as a threshold pass is omitted, default), >0: all pixels affect bloom brights more

floating-point number

N

camera/blade_count

Number of blades that make up the diaphragm mechanism. Used by depth_of_field.

integer

N

camera/fstop

Opening of the camera lens. Aperture is 1 / fstop with typical lens going down to f / 1.2 (larger opening). Used by depth_of_field and manual exposure mode.

floating-point number

N

camera/iso

Only for auto_exposure mode ‘manual’. The camera sensor sensitivity.

floating-point number

N

camera/min_fstop

Maximum aperture. Used by depth_of_field and manual exposure mode.

floating-point number

N

camera/shutter_speed

Only for auto_exposure mode ‘manual’. The camera shutter speed in 1/N seconds, N being the parameter value.

floating-point number

N

color_grading/color_contrast/[r g b a]

This will adjust the tonal range of light and dark color values in your scene. Lowering the intensity will remove highlights and lighten the image resulting in a washed-out appearance, whereas a higher intensity will tighten the highlights and darken the overall image.

floating-point number

N

color_grading/color_gain/[r g b a]

This will adjust the luminance intensity of the image’s whites (highlights) to accurately reproduce colors. Raising or lowering this value will result in the image highlights being washed-out or too dark.

floating-point number

N

color_grading/color_gamma/[r g b a]

This will adjust the luminance intensity of the image to accurately reproduce colors. Raising or lowering this value will result in the image mid-tones being washed-out or too dark.

floating-point number

N

color_grading/color_offset/[r g b a]

This will adjust the luminance intensity of the image’s blacks (shadows) to accurately reproduce colors. Raising or lowering this value will result in the image shadows being washed-out or too dark.

floating-point number

N

color_grading/color_saturation/[r g b a]

This will adjust the intensity (purity) of the colors (hue) that are being represented. A higher saturation intensity will result in colors appearing more like their purest forms (red, green, blue) and when saturation is lowered colors will appear more gray or washed-out.

floating-point number

N

color_grading/white_temp

White balance in relation to the temperature of the light in the scene. White light: when this matches light temperature. Warm light: When higher than the light in the scene, it is a yellowish color. Cool light: When lower than the light. Blueish color.

floating-point number

N

color_grading/white_tint

White balance temperature tint. Adjusts cyan and magenta color ranges. This should be used along with the white balance Temp property to get accurate colors. Under some light temperatures, the colors may appear to be more yellow or blue. This can be used to balance the resulting color to look more natural.

floating-point number

N

depth_of_field/blur_distance_km

Distance at which the pixel is blurred with a radius half of blur_radius. Expressed in km.

floating-point number

N

depth_of_field/blur_radius

Radius in pixels at 1080p resolution to apply according to distance from camera to emulate atmospheric scattering.

floating-point number

N

depth_of_field/focal_distance

Distance at which the depth of field effect should be sharp. Expressed in centimeters.

floating-point number

N

dirt_mask/dirt_texture

Name of the texture that defines the dirt on the camera lens where the light of very bright objects is scattered.

[default]

N

dirt_mask/intensity

Dirt mask intensity.

floating-point number

N

dirt_mask/tint/[r g b a]

Dirt mask tint.

floating-point number

N

gamma

Target gamma value of the camera.

floating-point number

N

image_effects/grain_intensity

Intensity of the grain effect.

floating-point number

N

image_effects/grain_jitter

Jitter level on the grain effect.

floating-point number

N

image_effects/vignette_intensity

Vignetting effect intensity.

floating-point number

N

lens_flare/corona_rays1/angle

Orientation of the corona in radians.

floating-point number

N

lens_flare/corona_rays1/intensity

Corona intensity.

floating-point number

N

lens_flare/corona_rays1/length

Length of each corona streak.

floating-point number

N

lens_flare/corona_rays1/spread

Value that defines how spread out the corona streaks are.

floating-point number

N

lens_flare/corona_rays1/streaks

Number of corona streaks.

integer

N

lens_flare/corona_rays2/angle

Orientation of the corona in radians.

floating-point number

N

lens_flare/corona_rays2/intensity

Corona intensity.

floating-point number

N

lens_flare/corona_rays2/length

Length of each corona streak.

floating-point number

N

lens_flare/corona_rays2/spread

Value that defines how spread out the corona streaks are.

floating-point number

N

lens_flare/corona_rays2/streaks

Number of corona streaks.

integer

N

lens_flare/enabled

Flag enabling the lens flare effect.

boolean

N

lens_flare/ghost1/brightness

Ghost brightness.

floating-point number

N

lens_flare/ghost1/offset

Ghost offset relative to the sun position.

floating-point number

N

lens_flare/ghost1/shape

Ghost shape. Default value is pentagon.

[circle, pentagon, hexagon, octogon]

N

lens_flare/ghost1/size

Ghost size.

floating-point number

N

lens_flare/ghost2/brightness

Ghost brightness.

floating-point number

N

lens_flare/ghost2/offset

Ghost offset relative to the sun position.

floating-point number

N

lens_flare/ghost2/shape

Ghost shape. Default value is circle.

[circle, pentagon, hexagon, octogon]

N

lens_flare/ghost2/size

Ghost size.

floating-point number

N

lens_flare/ghost3/brightness

Ghost brightness.

floating-point number

N

lens_flare/ghost3/offset

Ghost offset relative to the sun position.

floating-point number

N

lens_flare/ghost3/shape

Ghost shape. Default value is circle.

[circle, pentagon, hexagon, octogon]

N

lens_flare/ghost3/size

Ghost size.

floating-point number

N

lens_flare/ghost4/brightness

Ghost brightness.

floating-point number

N

lens_flare/ghost4/offset

Ghost offset relative to the sun position.

floating-point number

N

lens_flare/ghost4/shape

Ghost shape. Default value is circle.

[circle, pentagon, hexagon, octogon]

N

lens_flare/ghost4/size

Ghost size.

floating-point number

N

lens_flare/halo/amplitude

Value that defines how spread out the halo is.

floating-point number

N

lens_flare/halo/intensity

Halo intensity.

floating-point number

N

lens_flare/halo/light_diffraction_offset

Value that defines how light diffraction interact with the halo.

floating-point number

N

lens_flare/halo/offset

Halo offset relative to its center.

floating-point number

N

lens_flare/halo/pos_x

Halo position in X.

floating-point number

N

lens_flare/sky_depth

Sky depth in centimeters. Objects masking the Sun and located below this distance will attenuate or eliminate the lens flare effect.

floating-point number

N

lens_flare/sun_disk/intensity

Sun disk intensity.

floating-point number

N

lens_flare/sun_disk/size

Sun disk size.

floating-point number

N

lens_flare/sun_rays/sun_light_diffraction_intensity

Sun light diffraction intensity.

floating-point number

N

lens_flare/sun_rays/sun_light_diffraction_offset

Sun light diffraction offset.

floating-point number

N

lens_flare/sun_rays/sun_light_diffraction_threshold

Sun light diffraction threshold.

floating-point number

N

lens_flare/sun_rays/sun_solar_wind_speed

Sun solar wind speed.

floating-point number

N

lens_flare/tint_color/[r g b a]

Tint color.

floating-point number

N

motion_blur/intensity

Strength of motion blur [0.0 , 1.0].

floating-point number

N

motion_blur/max_distortion

Max distortion caused by motion blur. Percentage of screen width.

floating-point number

N

motion_blur/min_object_screen_size

Percentage of screen width objects must have for motion blur, lower value means less draw calls.

floating-point number

N

motion_blur/target_fps

Target frame per second for motion blur.

integer

N

override

If true (default), override the post-process configuration coming with the scene.

boolean

N

tonemapper/black_clip

Set where the crossover happens and white tones start to cut off their value. Subtle change in most cases [0.0, 1.0].

floating-point number

N

tonemapper/shoulder

Adjusts bright color in the tonemapper [0.0, 1.0].

floating-point number

N

tonemapper/slope

Steepness of the S-curve for the tonemapper. Larger values make the slope steeper (darker) [0.0, 1.0].

floating-point number

N

tonemapper/toe

Adjusts dark color in the tonemapper [0.0, 1.0].

floating-point number

N

tonemapper/white_clip

This should NOT be adjusted. Sets where the crossover happens and black tones start to cut off their value [0.0, 1.0].

floating-point number

N

recording/

Name

Description

Expected type

Read-only

enable

Start/stop the recording of frames into separate files

boolean

N

format

Format of the recording frame files. Default is png.

[png, hdr]

N

path

Directory path where the frames are saved. Defaults to the current directory.

string

N

wide_angle_lens/

Name

Description

Expected type

Read-only

coeffs

Comma-separated list of coefficients for the mapping function. Some mapping functions need one or more coefficients to work. Warning: No spaces are allowed in the list.

string

N

cutoff_angle

Expressed in degrees. Everything outside of the specified angle will be hidden.

floating-point number

N

cx

X principal point [0..1].

floating-point number

N

cy

Y principal point [0..1].

floating-point number

N

debug

Internal use. Renders the debug texture.

boolean

N

enabled

Flag enabling the wide-angle lens effect.

boolean

N

env_texture_size

Resolution of the environment cube map used to draw the world.

floating-point number

N

mapping_function

Name of the mapping function.

[equidistant, stereographic, equisolid_angle, orthographic, kumler_bauer]

N