Rspress

# #S600 ONNX Operator BPU Constraint List

In the table below:

lhs: left-hand side, the left-hand side in the operation.

rhs: right-hand side, the right-hand side in the operation.

Note

1. The DeformConv operator itself has certain special characteristics. We have implemented direct support for `torchvision::deformconv` through a specific custom operator method. Please contact Horizon for more information.
2. The GroupNormalization operator, when exported from the Pytorch framework to ONNX, will be automatically split by the framework into multiple suboperators into the original ONNX model, and we support BPU computation for these suboperators, thus implementing support for the GroupNormalization operator.

ONNX Operator Name | BPU Support Constraints
--- | ---
Abs | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Acos | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Acosh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Add | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
And | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
ArgMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDim
ArgMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 32767] Element : if type is int8, int16 or float16, reduce Elements size ∈ [1, 65535] dims: If type is int32, float32, only support one dim. output: Type: int8, int16, int32 Shape: reduce dim will be 1 or fused depend on keepDim
Asin | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Asinh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Atan | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Atanh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
AveragePool | input: Type: int8, int16 Shape: [*,H,W,C] or [*,L,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] or [SL] Dim: SH, SW, SL ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] or [PL_BEGIN,PL_END] PH_BEGIN,PW_BEGIN,PL_BEGIN,PH_END,PW_END,PL_END ∈ [-255, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1}
BatchNormalization | N/A, collapsed in graph optimization phase
BitwiseAnd | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseNot | input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseOr | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseXor | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Cast | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. Not support combinations (float16,float32→bool8). Shape: [*]
Ceil | if qtype is int8 input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
otherwise input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Celu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Clip | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Concat | input: Type: No limits Arg Number: input number ∈ [1, 1024] Dim: all dims < 131072 Size: size < 2G output: The input and output types need to be the same. Dim: all dims < 131072 Size: size < 2G
Constant | N/A, collapsed in graph optimization phase
ConstantOfShape | N/A, collapsed in graph optimization phase
Conv | input: --conv 1d-- Type: int8, int16 Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32, float32, float16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32, float32, float16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32, float32, float16 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum)
ConvTranspose | input: --convtranspose 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [*,W,C] 1d_Dim: * ∈ [1, 128]; W ∈ [1, 65536]; C ∈ [1, 2048] 2d_Shape: [*,H,W,C] 2d_Dim: * ∈ [1, 128]; H,W ∈ [1, 65536]; C ∈ [1, 2048] --convtranspose 3d-- Type: int8 3d_Shape: [*,D,H,W,C] 3d_Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 2048] weight: --convtranspose 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [N,KW,C] 1d_Dim: N,C ∈ [1, 2048]; KW ∈ [1, 14] 1d_Size: KW × C ∈ [1, 65535] 2d_Shape: [N,KH,KW,C] 2d_Dim: N,C ∈ [1, 2048]; KH,KW ∈ [1, 14]; KH,KW cannot both be 1 2d_Size: KH × KW × C ∈ [1, 65535] --convtranspose 3d-- Type: int8 3d_Shape: [N,KD,KH,KW,C] 3d_Dim: N,C ∈ [1, 2048]; KD,KH,KW ∈ [1, 14]; KD,KH,KW cannot all be 1 3d_Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: Type: int8, int16, int32, float16, float32 stride: 1d_Shape: [SW] 1d_Dim: SW ∈ [1, 14]; 2d_Shape: [SH,SW] 2d_Dim: SH,SW ∈ [1, 14]; 3d_Shape: [SD,SH,SW] 3d_Dim: SD,SH,SW ∈ [1, 14]; pad: 1d_Shape: [P_left,P_bottom] 1d_Dim: P_left,P_bottom ∈ [0, 256] 2d_Shape: [P_top,P_left,P_bottom,P_right] 2d_Dim: P_top,P_left,P_bottom,P_right ∈ [0, 256] 3d_Shape: [P_front,P_top,P_left,P_back,P_bottom,P_right] 3d_Dim: P_front,P_top,P_left,P_back,P_bottom,P_right ∈ [0, 256] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: 1d_Shape: [DW] 1d_Dim: DW ∈ {1} 2d_Shape: [DH,DW] 2d_Dim: DH,DW ∈ {1} 3d_Shape: [DD,DH,DW] 3d_Dim: DD,DH,DW ∈ {1}
Cos | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Cosh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
CumSum | input: Type: int8, int16, float16, float32 Shape: [*, dim[axis], *] Dim: * ∈ [1, 65536]; dim[axis] ∈ [1, 8192] output: Type: int8, int16, int32, float16, float32. only support combinations (int8→int8,int16,int32), (int16→int16,int32), (float16→float16), (float32→float32) The Shape and Dim is same as input exclusive: If type is float16, float32, only support exclusive == 0, otherwise, exclusive is 0 or 1 reverse: If type is float16, float32, only support reverse == 0, otherwise, exclusive is 0 or 1
DeformConv | input: Type: int8 Shape: [*,H,W,C] Dim: H,W ∈ [1, 1024]; H × W ≤ 720 × 1024; other dims ∈ [1, 65536] offset: Type: int16 Shape: [*,OH,OW,2 × offsetGroupNum × KH × KW] Size: 2 × offsetGroupNum × KH × KW ∈ [2, 256], OH × KH × OW × KW ≤ 720 × 1024 mask: Type: int8 Shape: [*,OH,OW,offsetGroupNum × KH × KW] Size: offsetGroupNum × KH × KW ∈ [1, 128] The value of mask is usually [0, 1] weight: Type: int8 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 8]; N ∈ [1, 4096] Size: KH × KW × C ∈ [1, 65535] bias: Type: float32 output: Type: int8, int16, int32 Other constraints: Same as fin stride: Shape: [SH,SW] Dim: SH,SW ∈ [1] pad: Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ [-H/2, 256], P_left,P_right ∈ [-W/2, 256] groupNum: Fin.c is divisible by group number offsetGroupNum: Fin.c is divisible by offset group number Size: offsetGroupNum ∈ [1, 2] dilation: Shape: [DH,DW] Dim: DH,DW ∈ [1] others: For each group, fin.c ∈ [1, 8192], KH × KW × fin.c ∈ [1, 65535], fin.c = C when group = 1
DepthToSpace | input: Type: No limits output: The input and output types need to be the same.
Div | lhs: Type: quantized type support int8, int16, int32, others support int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: For quantized type, support (int8,int16,int32→int8,int16,int32), others the input and output types need to be the same. Shape: [*] rounding_mode: For integer, only support TRUNC, For float, only support NONE. others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Dropout | N/A, collapsed in graph optimization phase
Elu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Equal | lhs: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
Erf | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Exp | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Expand | input: Type: No limits output: The input and output types need to be the same.
EyeLike | input: Type: int8, int16, int32, float16, float32, bool8 Shape: [*] output: Type: int8, int16, int32, float16, float32, bool8. Not support combinations (float16,float32→bool8). Shape: [*]
Floor | if qtype is int8 input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
otherwise input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
GRU | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Type: The input and output types need to be the same.
Gather | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. index: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] index value should not be larger than 1048576 output: The input and output types need to be the same.
GatherElements | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. N × C should not be larger than 1048576 When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] indices value should not be larger than 32767 Indices will transpose to [N, D, C]. D is indicesShape[dim], N is the product of indicesShape[:dim], C is the product of indicesShape[dim+1:]. N, C ∈ [1, 1048576]. IndicesShape[i] <= inputShape[i] for all dimensions i != dim. output: The input and output types need to be the same.
GatherND | input: Type: int8, int16, int32, float16, float32 Shape: [*] If gather1d, W = inputShape[batchDim], when C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If gather2d, H = inputShape[batchDim], W = inputShape[batchDim+1], H ∈ [1, 65536], W ∈ [1, 1048576]. If input type is int32, float32, W ∈ [1, 65536]. B is product of inputShape[0: batchDim], B ∈ [1, 1048576]. C is product of inputShape[batchDim+D:], C ∈ [1, 1048576]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*, D] indices value should not be larger than 1048576. D ∈ [1, 2]. output: Shape: [*] The input and output types need to be the same. batchDim: The number of batch dimensions. The gather of indexing starts from dimension of input[batchDim:]
Gelu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Gemm | lhs: Type: int8, int16 Shape: [M,K] Dim: M,K ∈ [1, 8192] rhl: Type: int8, int16 Shape: [K,N] Dim: K ∈ [1, 8192], N ∈ [1, 1048576] output: Type: int8, int16, int32, float16, float32 Shape: [M,N] Other constraints: Same as lhs and rhs
GlobalAveragePool | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim.
GlobalMaxPool | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16,  reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
Greater | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
GreaterOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
GridSample | input: Type: int8, int16, float16 Shape: [*,H,W,C] Dim: If type is int8, H,W ∈ [1, 32768], H and W cannot both be greater than 4096 at the same time, other dims ∈ [1, 65536]. If type is int16 or float16, H,W ∈ [1, 2048], H × W <= 1048576, other dims ∈ [1, 65536]. grid: Type: int16, float32. If gird type is float32, input type should be int16 or float16. Shape: [*,H,W,2] output: The input and output types need to be the same except Dim constraints mode: If input type is int8, support nearest and bilinear If input type is int16 or float16, only support bilinear. padding_mode: If input type is int8, support zeros and border. If input type is int16 or float16, only support zeros.
GroupNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
HardSigmoid | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
HardSwish | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Identity | N/A, collapsed in graph optimization phase
InstanceNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
LSTM | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Type: The input and output types need to be the same.
LayerNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
LeakyRelu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Less | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
LessOrEqual | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
Log | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
LogSoftmax | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
MatMul | lhs: Type: int8, int16 Shape: [*,M,C] Dim: * ∈ [1, 4096], M,C ∈ [1, 8192] rhl: Type: int8, int16 Shape: [*,C,N] Dim: * ∈ [1, 4096]; C ∈ [1, 8192], N ∈  [1, 1048576] output: Type: int8, int16, int32, float16, float32 Shape: [*,M,N] Other constraints: Same as lhs and rhs
Max | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
MaxPool | input: Type: int8, int16, float16 Shape: [*,H,W,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] Dim: SH, SW ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] PH_BEGIN,PW_BEGIN,PH_END,PW_END ∈ [-4, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1}
Min | lhs: Type: int8, int16, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Mish | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Mod | lhs: Type: int16, int32. Not support quantized type. Shape: [*] Value range: must be none-negative. rhs: The lhs and rhs types need to be the same. Shape: [*] Value range: must be positive. output: The input and output types need to be the same. Shape: [*]
Mul | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (int32→int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Neg | input: Type: int8, int16, int32, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Not | input: Type: int8, int16, bool8, float16 Shape: [*] output: Type: bool8 Shape: [*]
Or | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
PRelu | lhs: Type: int8, int16 Shape: [*] rhs: The type and shape is same as lhs output: The type and shape is same as lhs
Pad | N/A, collapsed in graph optimization phase
Pow | if exponent is an initializer with a single element 2, and the qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if exponent is an initializer with a single element 2, and the qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if exponent is an initializer with a single element 2, and the qtype is float16: input: Type: float16 output: Type: float16
Reciprocal | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
ReduceL2 | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
ReduceMax | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16,  reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
ReduceMean | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim.
ReduceMin | input: Type: int8, int16, int32, float16, float32 Shape: [*] Dim: if type is int8, int16 or float16, reduce axis dim size ∈ [1, 65535] Element : if type is int8, int16 or float16,  reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
ReduceSum | input: Type: int8, int16, float16, float32 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32), (float32→float32) Shape: reduce dim will be 1 or fused depend on keepDim dims: If type is float32, only support one dim.
Relu | input: Type: int8, int16, int32, float16, float32, if type is int32 or float32, this op must be fusible to a Conv op Shape: [*] output: The input and output types need to be the same. Shape: [*]
Reshape | input: Type: No limits output: The input and output types need to be the same.
Resize | input: --case 1: int8-- Type: int8 Shape: [*,H,W,C] --case 2: float16, int16-- Type: float16, int16(with per tensor quant info) Shape: [*,H,W,C] Dim: When steps ∈ (0, 1), and steps value are all an integer power of 2, batch * C ∈ [1, 512], other dims ∈ [1, 65536]. When steps value has one not an integer power of 2, H ∈ [1, 32768], W ∈ [1, 32768], C ∈ [1, 65536], other dims ∈ [1, 16384]. output: The input and output types need to be the same. mode: If Type is int8, support nearest and bilinear If Type is float16 or int16, only support bilinear padValue: If Type is int8, when padvalue is not equal to 0 and quantized, input only supports per tensor quantization step: If Type is int8, the integer part of step ∈ [-256, 255] If Type is float16 or int16, step ∈ [-8, 8] expansionMode: If Type is int8, support border and constant If Type is float16 or int16, only support border others: When type is float16 or int16, H and W cannot both be greater than 4096 at the same time. KN*KH*KW*KC*bitWidth/8 <= 1048576
Round | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16, float32 Shape: [*] decimals: Only support decimals=0. output: The input and output types need to be the same. Shape: [*]
ScatterElements | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: data and indices must be the same shape, except for scatterAxis updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min]
ScatterND | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: [*,C] Dim: *, C ∈ [1, 2] updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min]
Selu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Shape | N/A, collapsed in graph optimization phase
Sigmoid | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sign | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 Shape: [*] output: The shape and type is same as input
Sin | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sinh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Size | N/A, collapsed in graph optimization phase
Slice | if starts/ends/axes/steps are constant: input: Type: No limits Dim: all dims < 2097152 output: Type: The output datatype must be the same as the input Dim: all dims < 2097152
Softmax | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
Softplus | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
SpaceToDepth | input: Type: No limits output: The input and output types need to be the same.
Split | input: Type: No limits Dim: all dims < 2097152 output: The input and output types need to be the same. The input and output constraints are the same.
Sqrt | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sub | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Sum | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Tan | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Tanh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
ThresholdedRelu | input: Type: int8, int16, int32, float16, float32 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Tile | input: Type: No limits output: The input and output types need to be the same.
TopK | input: Type: int8, int16, int32, float16, float32 output: The type of output value is same as input, and indices is integer type. k: K <= 1024 others: The combined size in bytes of the op's operands and outputs must be no more than 4.8MB.
Transpose | input: Type: No limits output: The input and output types need to be the same.
Trilu | lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: int8, int16, int32, float16, float32. only support (int8,int16→int8,int16,int32), (int32→int32), (float16→float16), (float32→float32). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Where | condition: Type: bool8 Shape: [*] lhs: Type: int8, int16, int32, float16, float32 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Xor | lhs: Type: int8, int16, bool8, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]