ni_quadra_xstack
The ni_quadra_xstack filter mixes up to 50 streams together with or without alpha blending. The stacking of inputs is typically used to create a grid for broadcast streams used in applications like Zoom or MS Teams.
Prerequisites
The ni_quadra_xstack filter is unique in that it requires multiple input frames to produce an output. All inputs must be hardware frames on the same device. This imposes the following rules:
- If sources are from several decoding sessions:
Decoder must have explicit decode ID ie. (-c:v h264_ni_quadra_dec -dec 3 -xcoder-params 'out=hw' -i input1.h264 -c:v vp9_ni_quadra_dec -dec 3 -xcoder-params 'out=hw' -i input2.ivf …) - If sources are from upload and decoding sessions:
Decoder and upload must have same ID ie. (-c:v h264_ni_quadra_dec -dec 1 -xcoder-params 'out=hw' -i input1.h264 -c:v h264_ni_quadra_dec -dec -1 -xcoder-params 'out=sw' -i input2.h264 -filter_complex '[1:v]ni_quadra_hwupload=1[in2]; [0:v][in2]ni_quadra_xstack=0:0[out];…) - Multiple upload instances with ni_quadra_hwupload will also require the upload parameter to have matching device ID.
- Any input YUV (Y’CbCr) video should be limited range. Full range video will be accepted; however, the output video will be clamped to limited range before processing and will have a slight color shift.
Parameters
The ni_quadra_xstack parameters are mostly identical to the FFmpeg xstack but it uses the hardware to do the stacking. The inputs and output from the ni_quadra_xstack filter is a NETINT hardware AVFrame and the output frame rate will be the same as the frame rate as the first input except when explicitly specified by the sync parameter below. All inputs to the ni_quadra_xstack filters must have the same pixel format. For RGB pixel formats with alpha, the alpha must be straight alpha and not premultiplied (associated) alpha.
Below are the parameters for the ni_quadra_xstack filter:
inputs
Set the number of inputs that the filter will receive.
| Supported Values | Integer in the range 2 to 50 |
| Default | 2 |
layout
Specifies the layout for the x and y coordinates of the input videos on the output video. This option requires the desired layout configuration to be explicitly set by the user. This sets the position of each video input in output. Each input is separated by a ’|’. The first number represents the column, and the second number represents the row. Numbers start at 0 and are separated by a ’_’. Optionally one can use wX and hX, where X is video input from which to take width or height. Multiple values can be used when separated by a ’+’. In such case values are summed together. Odd values are rounded up to the nearest even integer.
Note that if inputs are of different sizes gaps may appear, as not all of the output video frame will be filled. Similarly, videos can overlap with each other if their position doesn’t leave enough space for the full frame of adjoining videos.
| Supported Values | Integer in the range 0 to 8192 |
| Default | For 2 inputs, a default layout of 0_0|w0_0 is set. In all other cases, a layout must be set by the user. |
size
Set the expression for the width and height of the input videos on the output video. If the input video is a different size than the output size specified by this parameter, then it is scaled to the output size. Each input is separated by a ’|’. The first number represents the width, and the second number represents the height. Numbers start at 1 and are separated by a ’_’. Odd values are rounded up to the nearest even number.
| Supported Values | Integer in the range 1 to 8192 |
| Default | Use input video width and height |
shortest
Force the output to terminate when the shortest input terminates.
| Supported Values | 0: Disable 1: Enable |
| Default | 0 |
fill
Fill the background of the output video with this specified color value
| Supported Values | RGBA in hex or an FFMpeg color name |
| Default | black |
sync
Use the specified input source as the primary source to synchronize all the other inputs with and generate an output with the same FPS as the specified input.
| Supported Values | - Integer in the range 0 to 49. - However, if the value specified is greater than or equal to the inputs parameter above, then input 0 will be used. |
| Default | 0 |
is_p2p
Specifies if output buffer of the filter is p2p buffer. When is_p2p is set to 1, the output buffer of the filter is set as p2p buffer and can be read out by p2p read. Else the output buffer is normal and cannot perform p2p.
| Supported Values | Bool 0 and 1 |
| Default | 0 |
keep_alive_timeout
Specifies a session keep alive timeout value. This is a periodic request/response between libxcoder and XCoder firmware that when timed out, terminates the session instance in the XCoder firmware.
| Supported Values | Integer in the range 1 to 100 |
| Default | 3 |
buffer_limit
Whether to limit the number of output buffers. If enabled, the buffering count of the filter is its buffer pool size(default 4) plus encoding buffering count if its downstream is encoder. It’s enabled by default in FFmpeg-n6.1, FFmpeg-n7.1, FFmpeg 7.1.3, FFmpeg-n8.0.1 and FFmpeg-n8.1.
| Supported Values | Integer 0 and 1 |
| Default | 0 |
When using ni_quadra_xstack in ffmpeg command line, if some of inputs frame rate are different, need to set –vsync 2 in ffmpeg command line to compatible with different frame rate outputs. For muxer with this extension they will select video sync as VSYNC_VFR by default
.avi .mkv .webm .flv .ts .m2t .m2ts .mts webp .apng .gif
The following is a Quadra xstack example. The decoder on device 0 decodes the h264 file Crowdrun_3840x2160p30_300.h264 and outputs hardware frames. The decoder on device 0 decodes the file Dinnerscene_1906x984p60_300.h264 and outputs hardware frames.
The two decoder outputs are inputs to the ni_quadra_xstack filter. The ni_quadra_xstack filter will re-size the first input to 320x240 and the second input to 320x240, as defined by the size parameter. After re-sizing both inputs, the ni_quadra_xstack filter will place the first input at position 0,0 and the second input at position 0,240 as defined by the layout parameter.
Finally, the output of the ni_quadra_stack filter is given to the NetInt h264 encoder which generates the file xstack_02.mkv
ffmpeg -y -loglevel debug -c:v h264_ni_quadra_dec -dec 0 -xcoder-params "out=hw" -i Crowdrun_3840x2160p30_300.h264 -c:v h264_ni_quadra_dec -dec 0 -xcoder-params "out=hw" -i Dinnerscene_1906x984p60_300.h264 -filter_complex "[0:v][1:v]ni_quadra_xstack=inputs=2:layout=0_0|0_h0:size=320_240|320_240[out]" -map "[out]" -c:v h264_ni_quadra_enc -enc 0 xstack_02.mkv
Please note that this filter is only available with FFmpeg-n4.1.3 and higher version since the native FFmpeg xstack filter was introduced in FFmpeg-n4.1.