List of Supported Operators for NPU Acceleration

FuriosaAI NPU and SDK can accelerate the following operators, as supported by Tensorflow Lite model and ONNX.

The names of the operators use ONNX as a reference.

Note

If NPU acceleration is not supported, it will run on the CPU. For some operators, even if NPU acceleration is supported, if certain conditions are not met, they may be split into several operators or may run on the CPU. Some examples of this would be when the weight of the model is larger than NPU memory, or if the NPU memory is not sufficient to process a certain computation.

Operators Accelerated on NPU

Name of operator

Additional details

Add

AveragePool

BatchNormalization

Acceleration supported, only if after Conv

Clip

Concat

Acceleration supported, only for height axis

Conv

Acceleration supported, only for group <= 128 and dilation <= 12

ConvTranspose

DepthToSpace

Exp

Expand

Flatten

Gemm

LeakyRelu

LpNormalization

Acceleration supported, only for p = 2 and batch <= 2

MatMul

MaxPool

Mean

Mul

Pad

ReduceL2

ReduceSum

Relu

Reshape

Pow

SpaceToDepth

Acceleration supported, only for mode=”CRD” and Furiosa SDK version 0.6.0 or higher

Sigmoid

Slice

Acceleration supported, only for height axis

Softmax

Acceleration supported, only for batch <= 2

Softplus

Sub

Split

Sqrt

Transpose

Unsqueeze