ni_quadra_overlay
The ni_quadra_overlay filter mixes two streams together with or without alpha blending, the main image and the overlay image. Overlays are typically used to overlay broadcast streams with logos or text. These typically use alpha blending, where the logo background would be transparent, or partially transparent to blend with the main image. Overlays can also be used for picture in picture which will typically not use alpha blending. An overlay can be a moving picture or a single still frame. Alpha defines how the weighting should be given to the color components during the blending process. The ni_quadra_overlay filter is supported for yuv420p, yuv420p10le, nv12, p010le, rgba, argb, abgr, bgra, bgr0, nv16, yuyv422 and uyvy422.
The ni_quadra_overlay filter only supports limited color range video (also known as “TV” range). While full range (or “PC” range) video is accepted, the ni_quadra_overlay filter will clamp to limited color range video before processing. This clamping will cause a slight color shift in the output video.
Prerequisites
The ni_quadra_overlay filter is unique in that it requires two input frames to produce an output. Both inputs must be hardware frames on the same device. This imposes the following rules:
- If sources are from two decoding sessions:
Decoder must have explicit decode ID ie
-c:v h264_ni_quadra_dec -dec 3 -xcoder-params 'out=hw' -i input1.h264 -c:v vp9_ni_quadra_dec -dec 3 -xcoder-params 'out=hw' -i input2.ivf …
- If sources are from upload and decoding session:
Decoder and upload must have same ID ie.
-c:v h264_ni_quadra_dec -dec 1 -xcoder-params 'out=hw' -i input1.h264 -c:v h264_ni_quadra_dec -dec -1 -xcoder-params 'out=sw' -i input2.h264 -filter_complex '[1:v]ni_quadra_hwupload=1[in2]; [0:v][in2]ni_quadra_overlay=0:0[out];…
- Multiple upload instances with ni_quadra_hwupload will also require the upload parameter to have matching device ID.
Parameters
The ni_quadra_overlay parameters are mostly identical to the FFmpeg overlay but it uses the hardware to do the overlay. The input and output from the ni_quadra_overlay filter is a NETINT hardware AVFrame and the underlying pixel format of the output will be the same as the main image. There is a special case where the output is changed to nv12 if the main is compressed 8-bit tiled 4x4 and overlay is rgb.
Note that the eof_action parameter defaults to repeat, this permits a single image to be overlaid for the entire duration of the main video stream.
Below are the parameters for the ni_quadra_overlay filter:
x y
Set the expression for the x and y coordinates of the overlaid video on the main video. In case the expression is invalid, it is set to a big value (meaning that the overlay will not be displayed within the output visible area). The x and y expressions can contain the following parameters, main_w, main_h, width and height of main image, overlay_w, overlay_h, width and height of overlay, hsub, vsub, horiz and vert chroma subsample values of the output, t (timestamp expressed in seconds).
| Supported Values | Integer in the range 0 to 8192 |
| Default | 0 for both expressions |
eof_action
The action to take when EOF is encountered on the secondary input. Repeat, repeats the last frame, endall ends both streams, pass continues with just the main stream.
| Supported Values | - endall - pass - repeat |
| Default | repeat |
shortest
Force the output to terminate when the shortest input terminates.
| Supported Values | - 0: Disable - 1: Enable |
| Default | 0: Disable |
repeatlast
Force the filter to extend the last frame of secondary streams until the end of the primary stream.
| Supported Values | - 0: Disable - 1: Enable |
| Default | 1: Enable |
alpha
Set format of alpha of the overlaid video, it can be straight or premultiplied. If the overlaid video is YUV, the overlay will completely overwrite the background because there is no alpha channel. If the overlaid video is an RGBA icon, then the overlay will blend with the background as per the alpha value of each pixel.
| Supported Values | - straight - premultiplied |
| Default | straight |
inplace
Perform an in-place overlay. The ni_quadra_overlay filter normally makes a copy of the background frame and applies the overlay to the copied frame. This copy operation can have a performance penalty on the Quadra device. To improve performance, the inplace parameter will apply the overlay immediately to the background frame. Although this improves performance, this restricts how you can use FFmpeg. You must not use the split or ni_quadra_split filter before the ni_quadra_overlay filter. Doing so may cause the video of other entities using the split to include the overlay. Also, if the overlay image is too large, performance can actually decrease.
| Supported Values | Bool 0 and 1 |
| Default | 0 |
is_p2p
Specifies if output buffer of the filter is p2p buffer. When is_p2p is set to 1, the output buffer of the filter is set as p2p buffer and can be read out by p2p read. Else the output buffer is normal and cannot perform p2p. This option won’t work when inplace is set to 1.
| Supported Values | Bool 0 and 1 |
| Default | 0 |
keep_alive_timeout
Specifies a session keep alive timeout value. This is a periodic request/response between libxcoder and XCoder firmware that when timed out, terminates the session instance in the XCoder firmware.
| Supported Values | Integer in the range 1 to 100 |
| Default | 3 |
buffer_limit
Whether to limit the number of output buffers. If enabled, the buffering count of the filter is its buffer pool size(default 4) plus encoding buffering count if its downstream is encoder. It’s enabled by default in FFmpeg-n6.1, FFmpeg-n7.1, FFmpeg 7.1.3, FFmpeg-n8.0.1 and FFmpeg-n8.1.
| Supported Values | Integer 0 and 1 |
| Default | 0 |
dw dh
Set the destination width and height. The overlay is resized to the destination width and height prior to placement on the background video. These two parameters only take effect when inplace=1 and the overlay is not rgba (or a swizzle equivalent like bgra) over a yuv background.
If inplace=0, the dw and dh parameters are ignored. If overlaying rgba (or swizzle variation) over yuv, the dw and dh parameters will also be ignored.
| Supported Values | Positive integer less than the background width/height. |
| Default | overlay width and height. |
The following is a Quadra overlay example. The decoder on device 0 decodes the background frames from the mp4 file. The icon overlay is a PNG file that is converted to RGBA, scaled, then transferred to the same hardware device using ni_quadra_hwupload. This icon will be overlaid on every frame because the default eof_action is repeat. The ni_quadra_overlay output is sent to the encoder which is also on device 0.
ffmpeg -dec 0 -c:v h264_ni_quadra_dec -xcoder-params "out=hw" -i WorldFootball-2min.mp4 -i cbctrans.png -filter_complex "[1:v]format=rgba,ni_quadra_hwupload=0[a];[0:v][a]ni_quadra_overlay=main_w-overlay_w-25:main_h-overlay_h-25[b]" -c:a copy -map "[b]" -enc 0 -c:v h265_ni_quadra_enc -xcoder-params "RcEnable=1:bitrate=2000000" -map "0:a" WorldFootball-overlay.mp4