Monitoring Device Load
A few utility programs are provided to list and monitor resource usage. Running the utility /usr/local/bin/ni_rsrc_list produces results showing capabilities of cards on the host. Another utility is /usr/local/bin/ni_rsrc_mon, that actively monitors the resource usage on the server and initializes resources.
To monitor the processing load of Quadra devices use the NETINT resource monitor utility called ni_rsrc_mon
ni_rsrc_mon
A sample output is shown below:
ni_rsrc_mon
Example
**************************************************
1 devices retrieved from current pool at start up
Fri May 29 11:58:40 2026 up 00:00:00 v---6t7DV
INDEX TEMP POWER FLAVOR FR SN
0 40 6922mW D ---6rxDV Q1A10BA11FC060-0044
Num decoders: 1
INDEX LOAD MODEL_LOAD INST MEM SHARE_MEM P2P_MEM DEVICE
0 0 0 0 0 0 0 /dev/nvme1n1
Num encoders: 1
INDEX LOAD MODEL_LOAD INST MEM SHARE_MEM P2P_MEM DEVICE
0 0 0 0 0 0 0 /dev/nvme1n1
Num scalers: 1
INDEX LOAD MODEL_LOAD INST MEM SHARE_MEM P2P_MEM DEVICE
0 0 0 0 0 0 0 /dev/nvme1n1
Num AIs: 1
INDEX LOAD MODEL_LOAD INST MEM SHARE_MEM P2P_MEM DEVICE
0 0 0 0 0 0 0 /dev/nvme1n1
**************************************************
The number of devices having encoders, decoders, scalers (2D Engines), and AI engines are listed.
The output format also shows the real-time temperature, power, flavor, current Firmware revision and Serial number.
To see the number of uploaders and firmware/system load by subsystem, try the full output format.
The simple output format shows the maximum firmware/system load amongst the subsystems on the Quadra device.
A help text with description of how to use the program can be accessed with the command:
-n Specify reporting interval in one second interval. If 0 or no selection, report only once.
Default: 0
-R Specify if refresh devices on host in each monitor interval.
Default: 1
-o Output format. [text, simple, full, json, json1, json2, extra].
Default: text
-D Dump firmware logs to current directory. Default: 0(not dump fw log).
-C Dump firmware core reset logs to current directory.Default: 0(not dump fwlog).
-k Specify to dump which card's firmware logs.
Default: -1(dump fw log of all cards).
-r Initialize Quadra device regardless firmware release version to libxcoder version compatibility.
Default: only init cards with compatible firmware version.
-t Set timeout time in seconds for device polling. Program will exit with failure if timeout is reached without finding at least one device. If 0 or no selection, poll indefinitely until a Quadra device is found.
Default: 0
-S Skip init_rsrc.
-d Print detailed infomation for decoder/encoder in text and json formats.
-l Set loglevel of libxcoder API.
[none, fatal, error, info, debug, trace]
Default: info
-q Show QoS information - MODE, NS, ALLOW, OPMAX
MODE – QoS mode 0/1/2. Default:0
NS – QoS NS set in mode 1/2. Default:1
ALLOW – QoS allowance set in mode 1/2. Default:0
OPMAX – QoS overprovisioning max set in mode 2. Default:0
-h Open this help message.
-v Print version info.
Interpreting load values
ni_rsrc_mon utilizes 3 card load mechanisms when reporting:
- Current load - the most recent VPU load in 1 second. Reported under LOAD/VPU
- FW model load - FW calculates "estimated" VPU load based on the number of instances, resolutions, encoding parameters, etc. Reported under MODEL_LOAD
- FW load - the most recent CPU load in 1 second. FW calculates CPU load by (busy cycle count / total cycle count). Reported under FW_LOAD/FW
ni_rsrc_monreported LOAD is the higher value between 1. current load and 2. fw load to reflect the factor that is likely to become the bottleneck on the card, e.g.- when running 540p multiple instances encoding, CPU load is usually higher, so
ni_rsrc_monwill report FW load as the LOAD, which may be the reason why we are seeing different LOAD value reported by ni_rsrc_mon. - For reference, ni_rsrc_mon printout can be found in ni_rsrc_mon.c print_text function, which prints LOAD value with the following condition -
- when running 540p multiple instances encoding, CPU load is usually higher, so
sessionCtxt->load_query.current_load > sessionCtxt->load_query.fw_load) ? sessionCtxt->load_query.current_load : sessionCtxt->load_query.fw_load
To print all the stats (including FW load) via ni_rsrc_mon specify –o option with json, json1 or full
Reporting columns for text output format
| INDEX | index number used by resource manager to identify the resource |
| TEMP | current temperature (degrees Celsius) |
| POWER | current power(mW), N/A when query power not supported |
| FLAVOR | |
| FR | current Flavor, N/A when query flavor not supportedcurrent firmware revision |
| SN | serial number of the Quadra device |
The Error information for critical/warning temperature and Pcie-gen mismatch is shown as a message.
In Json format above information is displayed per codec
| INDEX | index number used by resource manager to identify the resource |
| LOAD | realtime load given in percentage. This value is max of VPU and FW load reported in full output format |
| MODEL_LOAD | estimated load given in percentage based on framerate and resolution |
| INST | number of job instances |
| MEM | usage of memory given in percentage by the subsystem |
| SHARE_MEM | usage of memory shared across subsystems on the same device |
| P2P_MEM | usage of memory by P2P |
| DEVICE | path to NVMe block device file handle |
Additional reporting columns for full output format
| QOS | show QoS information once - MODE, NS, ALLOW, OPMAX when enabled |
| VPU | same as LOAD in JSON outputs |
| FW | system load |
| TOTAL | same as MEM |
| CRITICAL | usage of memory considered critical |
| L_FL2V | last ran firmware loader 2 version |
| N_FL2V | nor flash firmware loader 2 version |
| FR | current firmware revision |
| N_FR | nor flash firmware revision |
The INST column also shows the maximum supported number of job instances in the form of INST/MAX_INST.
Reporting columns for JSON format
| NUMBER | Total number of Quadra cards present |
| INDEX | index number used by resource manager to identify the resource |
| LOAD | realtime load given in percentage. This value is max of VPU and FW load reported in full output format |
| MODEL_LOAD | estimated load given in percentage based on framerate and resolution |
| FW_LOAD | system load |
| INST | number of job instances |
| MAX_INST | maximum supported number of job instances |
| MEM | usage of memory given in percentage by the subsystem |
| CRITICAL_MEM | usage of memory considered critical |
| SHARE_MEM | usage of memory shared across subsystems on the same device |
| P2P_MEM | usage of memory by P2P |
| DEVICE | path to NVMe block device file handle |
| L_FL2V | last ran firmware loader 2 version |
| N_FL2V | nor flash firmware loader 2 version |
| FR | current firmware revision |
| N_FR | nor flash firmware revision |
| NUMA_NODE | linux numa node |
| PCIE_ADDR | linux Pcie address |
| TEMP | current temperature (degrees Celsius) |
| POWER | current power(mW), N/A when query power not supported |
| SN | serial number of the Quadra device |
| ERROR | Denotes temperature and pcie gen errors.Default:none |
| FLAVOR | current Flavor, N/A when query flavor not supported |
| QOS | show QoS information - MODE, NS, ALLOW, OPMAX for dec and enc when enabled |
The output of -o json is in standard JSON format containing all the above information.
The output of -o json1 and -o json2 is in standard JSON format but contains a subset of the above information.
Reporting columns for extra output format
| TEMP | realtime temperature given in degrees Celsius |
| POWER | realtime power(mw), print N/A when query power not supported |
| DEVICE | path to NVMe block device file handle |
Reporting columns for detail output
Use ni_rsrc_mon -d
For Decoder
| INDEX | index number used by resource manager to identify the instance |
| AvgCost | estimated load given in percentage based on framerate and resolution |
| FrameRate | framerate of current session |
| IDR | number of IDR frame it processes |
| InFrame | number of frames send to the decoder |
| OutFrame | number of frames get from the decoder |
| Width | width of decoder |
| Height | height of decoder |
| SID | session id |
| DEVICE | path to NVMe block device file handle |
| NAMESPACE | path to namespace block device file handle |
For Encoder
| INDEX | index number used by resource manager to identify the instance |
| AvgCost | estimated load given in percentage based on framerate and resolution |
| FrameRate | framerate of current session |
| IDR | number of IDR frame it processes |
| UserIDR | number of force IDR frame set by user |
| InFrame | number of frames send to the encoder |
| OutFrame | number of frames get from the encoder |
| BR | instantaneous bitrate of current encoder session |
| AvgBR | average bitrate of current encoder session |
| Width | width of encoder |
| Height | height of encoder |
| Format | input frame format send to encoder |
| SID | session id |
| DEVICE | path to NVMe block device file handle |
| NAMESPACE | path to namespace block device file handle |
When using both -d and -o json series parameters simultaneously, the output of is in standard JSON format.
ni_rsrc_list
This tool is to display information for NETINT hardware.
A help text with description of how to use the program can be accessed with the command:
-a Print includes info for uninitialized cards.
-h Display this help and exit.
-v Print version info.
-o Output format. [text, full, json]
Default: text
-l Set loglevel of libxcoder API.
[none, fatal, error, info, debug, trace]
Default: info