Command Line Tools
Through the command line tools, Furiosa SDK provides functions such as monitoring NPU device information, compiling models, and checking compatibility between models and SDKs. This section explains how to install and use each command line tool.
furiosa-toolkit
furiosa-toolkit provides a command line tool that enables users to manage and check the information of NPU devices.
furiosa-toolkit installation
To use this command line tool, you first need to install the kernel driver as shown in Driver, Firmware, and Runtime Installation. Subsequently, follow the instructions below to install furiosa-toolkit.
sudo apt-get install -y furiosa-toolkit
furiosactl instructions
The furiosactl command provides a variety of subcommands and has the ability to obtain information or control the device.
furiosactl <sub command> [option] ..
furiosactl info
After installing the kernel driver, you can use the furiosactl command to check whether the NPU device is recognized.
Currently, this command provides the furiosactl info command to output temperature, power consumption and PCI information of the NPU device.
If the device is not visible with this command after mounting it on the machine, Driver, Firmware, and Runtime Installation to install the driver.
If you add the --full option to the info command, you can see the device’s UUID and serial number information together.
$ furiosactl info
+------+--------+----------------+-------+--------+--------------+
| NPU | Name | Firmware | Temp. | Power | PCI-BDF |
+------+--------+----------------+-------+--------+--------------+
| npu1 | warboy | 1.6.0, 3c10fd3 | 54°C | 0.99 W | 0000:44:00.0 |
+------+--------+----------------+-------+--------+--------------+
$ furiosactl info --full
+------+--------+--------------------------------------+-------------------+----------------+-------+--------+--------------+---------+
| NPU | Name | UUID | S/N | Firmware | Temp. | Power | PCI-BDF | PCI-DEV |
+------+--------+--------------------------------------+-------------------+----------------+-------+--------+--------------+---------+
| npu1 | warboy | 00000000-0000-0000-0000-000000000000 | WBYB0000000000000 | 1.6.0, 3c10fd3 | 54°C | 0.99 W | 0000:44:00.0 | 511:0 |
+------+--------+--------------------------------------+-------------------+----------------+-------+--------+--------------+---------+
furiosactl list
The list subcommand provides information about the device files available on the NPU device.
You can also check whether each core present in the NPU is in use or idle.
furiosactl list
+------+------------------------------+-----------------------------------+
| NPU | Cores | DEVFILES |
+------+------------------------------+-----------------------------------+
| npu1 | 0 (available), 1 (available) | npu1, npu1pe0, npu1pe1, npu1pe0-1 |
+------+------------------------------+-----------------------------------+
furiosactl ps
The ps subcommand prints information about the OS process currently occupying the NPU device.
$ furiosactl ps
+-----------+--------+------------------------------------------------------------+
| NPU | PID | CMD |
+-----------+--------+------------------------------------------------------------+
| npu0pe0-1 | 132529 | /usr/bin/python3 /usr/local/bin/uvicorn image_classify:app |
+-----------+--------+------------------------------------------------------------+
furiosactl top (experimental)
The top subcommand is used to view utilization by NPU unit over time.
The output has the following meaning
By default, utilization is calculated every 1 second, but you can set the calculation interval yourself with the --interval option. (unit: ms)
Item |
Description |
|---|---|
Datetime |
Observation time |
PID |
Process ID that is using the NPU |
Device |
NPU device in use |
NPU(%) |
Percentage of time the NPU was used during the observation time. |
Comp(%) |
Percentage of time the NPU was used for computation during the observation time |
I/O (%) |
Percentage of time the NPU was used for I/O out of the time the NPU was used |
Command |
Executed command line of the process |
$ furiosactl top --interval 200
NOTE: furiosa top is under development. Usage and output formats may change.
Please enter Ctrl+C to stop.
Datetime PID Device NPU(%) Comp(%) I/O(%) Command
2023-03-21T09:45:56.699483936Z 152616 npu1pe0-1 19.06 100.00 0.00 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:56.906443888Z 152616 npu1pe0-1 51.09 93.05 6.95 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:57.110489333Z 152616 npu1pe0-1 46.40 97.98 2.02 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:57.316060982Z 152616 npu1pe0-1 51.43 100.00 0.00 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:57.521140588Z 152616 npu1pe0-1 54.28 94.10 5.90 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:57.725910558Z 152616 npu1pe0-1 48.93 98.93 1.07 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:57.935041998Z 152616 npu1pe0-1 47.91 100.00 0.00 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
2023-03-21T09:45:58.13929122Z 152616 npu1pe0-1 49.06 94.94 5.06 ./npu_runtime_test -n 10000 results/ResNet-CTC_kor1_200_nightly3_128dpes_8batches.enf
furiosa
The furiosa command is a meta-command line tool that can be used by installing the Python SDK <PythonSDK>.
Additional subcommands are also added when the extension package is installed.
If the Python execution environment is not prepared, refer to Python execution environment setup.
Installing command line tool.
$ pip install furiosa-sdk
Verifying installation.
$ furiosa compile --version
furiosa-compiler: 0.9.0 (rev: e626c458c built at 2023-04-19T13:57:11Z)
furiosa-tools: 0.9.0-release (rev: 474835a)
furiosa compile
The compile command compiles models such as TFLite and ONNX, generating programs that utilize FuriosaAI NPU.
Detailed explanations and options can be found in the furiosa compile page.
furiosa litmus (Checking for model compatibility)
The litmus command takes the TFLite and ONNX models as arguments,
quantizes them automatically, and attempts to compile up to the final binary, in order to check whether the given model is compatible with the SDK.
$ furiosa litmus foo.onnx
libfuriosa_hal.so --- v0.11.0, built @ 43c901f
INFO:furiosa.common.native:loaded native library libfuriosa_compiler.so.0.9.0 (0.9.0 e626c458c)
furiosa-quantizer 0.9.0 (rev. 474835a) furiosa-litmus 0.9.0 (rev. 474835a)
[Step 1] Checking if the model can be loaded and optimized ...
[Step 1] Passed
[Step 2] Checking if the model can be quantized ...
[Step 2] Passed
[Step 3] Checking if the model can be saved as a file ...
[Step 3] Passed
[Step 4] Checking if the model can be compiled for the NPU family [warboy-2pe] ...
[1/5] 🔍 Compiling from dfg to ldfg
▪▪▪▪▪ [1/3] Splitting graph(LAS)...Done in 7.1025434s
▪▪▪▪▪ [2/3] Lowering graph(LAS)...Done in 17.59427s
▪▪▪▪▪ [3/3] Optimizing graph...Done in 0.022468261s
Done in 24.721376s
[2/5] 🔍 Compiling from ldfg to cdfg
Done in 0.002730215s
[3/5] 🔍 Compiling from cdfg to gir
Done in 0.018330544s
[4/5] 🔍 Compiling from gir to lir
Done in 0.004659492s
[5/5] 🔍 Compiling from lir to enf
Done in 0.026905483s
✨ Finished in 24.77446s
[Step 4] Passed
Should it fail, you will see an error message like the one below. You can seek help by filing a bug report to FuriosaAI customer service center.
$ furiosa litmus bar.onnx
libfuriosa_hal.so --- v0.11.0, built @ 43c901f
INFO:furiosa.common.native:loaded native library libfuriosa_compiler.so.0.9.0 (0.9.0 e626c458c)
furiosa-quantizer 0.9.0 (rev. 474835a) furiosa-litmus 0.9.0 (rev. 474835a)
[Step 1] Checking if the model can be loaded and optimized ...
[Step 1] Failed
Traceback (most recent call last):
File "/.../python-3.10.6/bin/furiosa-litmus", line 8, in <module>
sys.exit(main())
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/litmus/__init__.py", line 172, in main
validate(Path(args.model_path), verbose=args.verbose, target_npu=args.target_npu)
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/litmus/__init__.py", line 121, in validate
raise e
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/litmus/__init__.py", line 115, in validate
onnx_model = optimize_model(onnx_model)
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/optimizer/frontend/onnx/__init__.py", line 24, in optimize_model
model = _transform([CheckVersion(opset_version).transform], model)
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/optimizer/frontend/onnx/__init__.py", line 37, in _transform
model = transform(model)
File "/.../python-3.10.6/lib/python3.10/site-packages/furiosa/optimizer/frontend/onnx/utils/version_checker.py", line 26, in transform
version = int(model.opset_import[0].version)
IndexError: list index (0) out of range