Skip to main content
Version: v5.9.0

Gstreamer Plugins

Starting from the Quadra release V4.7, NETINT provides support for GStreamer 1.22.2 and later versions, enabling Quadra transcode and filters functionality.

The GStreamer plugins directly use the NETINT libxcoder interface.

Decoding​

The list of GStreamer NETINT command options for decoding can be shown with this command:

gst-inspect-1.0 --plugin niquadra | grep decoder

The result of <decoder name> is as following：

  • niquadrah264dec: NetInt NIQUADRA H264 decoder
  • niquadrah265dec: NetInt NIQUADRA H265 decoder
  • niquadrajpegdec: NetInt NIQUADRA JPEG decoder
  • niquadravp9dec: NetInt NIQUADRA VP9 decoder

Get more details by following command:

gst-inspect-1.0 <decoder name>

Example:

gst-inspect-1.0 niquadrah264dec

Encoding​

The list of GStreamer NETINT command options for decoding can be shown with this command:

gst-inspect-1.0 --plugin niquadra | grep encoder

The result of <encoder name> is as following：

  • niquadraav1enc: NetInt QUADRA AV1 encoder
  • niquadrah264enc: NetInt QUADRA H264 encoder
  • niquadrah265enc: NetInt QUADRA H265 encoder
  • niquadrajpegenc: NetInt QUADRA JPEG encoder

Get more details by following command:

gst-inspect-1.0 <encoder name>

Example:

gst-inspect-1.0 niquadrah264enc

Filters​

The list of GStreamer NETINT command options for filters is as following:

  • niquadrabgr: NIBGR Netint element
  • niquadracrop: NICROP Netint element
  • niquadradrawbox: NIDRAWBOX Netint element
  • niquadrahwdownload: HWDownload Netint element
  • niquadrahwupload: HWUpload Netint element
  • niquadraoverlay: Overlay NETINT element
  • niquadrapad: NIPAD Netint element
  • niquadraroi: NIROI Netint element
  • niquadrarotate: NIROTATE Netint element
  • niquadrascale: NISCALE Netint element
  • niquadrastack: Stack NETINT element

Known issues.​

The following is a complete list of all known issues.

  1. Gstreamer with NETINT encoding produces different results from FFmpeg

The FFmpeg encoder does not contain the aspect ratio in VUI by default. Gstreamer will add an aspect ratio in the VUI with the default value ”1/1”. See the below pipeline for an example. Change the default value by using the pixel-aspect-ratio property.

gst-launch-1.0 filesrc location=/opt/work/FFmpegXcoder/libxcoder/test/akiyo_352x288p25.yuv ! videoparse width=352 height=288 format=i420 framerate=25/1 pixel-aspect-ratio=1/2 ! niquadrah265enc ! filesink location=~/aki-gstreamer-enc-ni.265

The FFmpeg encoder does not contain any color description in the VUI by default. If it is not explicitly specified choose a default colorimetry for GStreamer. The default colorimetry will be different for different resolutions. The test video is 355x288 which is SD, so the first BT601 is chosen. See the pipeline example below, this overwrites the default value by adding “video/x-raw,colorimetry=bt709”

gst-launch-1.0 filesrc location=/opt/work/FFmpegXcoder/libxcoder/test/akiyo_352x288p25.yuv ! videoparse width=352 height=288 format=i420 framerate=25/1 ! video/x-raw,colorimetry=bt709 ! niquadrah265enc ! filesink location=~/aki-gstreamer-enc-ni.265
  1. Gstreamer will not support wmv containers with h264 format
  2. Add limit to the processing speed of Gstreamer for 8k 10bit input video

Due to the limited memory pool size on the firmware, the processing speed of the filters with multiple HW frame inputs like overlay/xstack is generally controlled, especially for high resolution like 8k-10bit input video. For example we can set the property “max-size-buffers” and “max-size-bytes” for queue element to control the processing speed for the following test command:

gst-launch-1.0 filesrc location=/home/test/Shanghai_7680x4320p30_450_10bit.h264 ! h264parse ! niquadrah264dec xcoder-params='out=hw' dec=0 ! niquadraoverlay x=0 y=0 name=overlay ! tee name=t ! queue max-size-buffers=1 max-size-bytes=663552000 ! niquadrah264enc enc=0 ! h264parse ! filesink async=false location=/home/test/test1.h264 t. ! queue max-size-buffers=1 max-size-bytes=663552000 ! niquadrascale ! “video/x-raw(memory:NiQuadraMemory), width=959, height=539” ! niquadrah264enc enc=0 ! h264parse ! filesink async=false location=/home/test/test2.h264 t. ! queue max-size-buffers=1 max-size-bytes=663552000 ! niquadrascale ! “video/x-raw(memory:NiQuadraMemory), width=480, height=270” ! niquadrah264enc enc=0 ! h264parse ! filesink async=false location=/home/test/test3.h264 filesrc location=/home/test/Crowdrun_3840x2160p30_300.h264 ! h264parse ! niquadrah264dec xcoder-params='out=hw' dec=0 ! niquadrascale ! “video/x-raw(memory:NiQuadraMemory), width=960, height=540” ! overlay.

If the limited isn’t added, this test case will report “insufficient resource” issue because of insufficient memory.

  1. To better align with GStreamer’s design philosophy, the dimension properties have been removed from the following plugins. Output image dimensions must now be negotiated exclusively via caps:
  • niquadramerge
  • niquadrarotate
  • niquadraaipre
  • niquadracrop
  • niquadrapad
  • niquadrascale

Added features:​

  1. niquadrascale**:**

To scale the input video with the original aspect ratio retained, use the original-aspect-ratio property of this plugin. Once this feature is turned on (default is off), a dimension range, instead of a fixed value, needs to be specified in the downstream caps:

original-aspect-ratio=decrease:

keep the original aspect ratio and scale the input video to fit inside the box, whose upper bound and step (previous divisible-by property of the scaler) are specified by downstream Capsfilter. For example, to scale the input picture to fit in a 1920x1080 box with step 2, do the following:

...! niquadrascale original-aspect-ratio=decrease ! "video/x-raw(memory:NiQuadraMemory), width=[2,1920,2], height=[2,1080,2]" !...

original-aspect-ratio=increase:

keep the original aspect ratio and scale the input video to cover the box, whose lower bound and step (previous divisible-by property of the scaler) are specified by downstream Capsfilter. For example, to scale the input picture to cover a 1920x1080 box with step 2, do the following:

...! niquadrascale original-aspect-ratio=increase ! "video/x-raw(memory:NiQuadraMemory), width=[1920, 8192, 2], height=[1080, 8192, 2]" !...

Note: the third parameter for width and height is the step of which default value is 1**.**

  1. niquadrapad:

To put input video in the center of padded area, set x=-1 and y=-1. Note: the default values for x and y are 0s.