Rspress

# #J6B ONNX算子BPU约束列表

下方表格中：

lhs：left-hand side，指运算中的左操作数。

rhs：right-hand side，指运算中的右操作数。

注解

1. DeformConv算子本身存在一定特殊性， 我们通过特定的自定义算子的方法实现了对 `torchvision::deformconv` 的直接支持， 如有需要请联系地平线技术支持人员获取相关说明。
2. GroupNormalization算子在从Pytorch框架导出到ONNX时， 会被框架自动拆分成多个子算子到原始ONNX模型中， 我们支持对这些子算子的BPU计算，从而实现了对GroupNormalization算子的支持。

ONNX Operator Name | BPU Support Constraints
--- | ---
Abs | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
Acos | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Acosh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Add | lhs: Type: int8, int16, int32, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: If type is int32, this op must be fusible to a conv op, one input is conv output which is int32, the other input must be int8 or int16. When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
And | lhs: Type: int8, int16, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
ArgMax | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 32767] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, int32 The size of the reduced axis can be represented using an int16 number. Shape: reduce dim will be 1 or fused depend on keepDim
ArgMin | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 32767] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, int32 The size of the reduced axis can be represented using an int16 number. Shape: reduce dim will be 1 or fused depend on keepDim
Asin | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Asinh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Atan | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Atanh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
AveragePool | input: Type: int8, int16 Shape: [*,H,W,C] or [*,L,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] or [SL] Dim: SH, SW, SL ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] or [PL_BEGIN,PL_END] PH_BEGIN,PW_BEGIN,PL_BEGIN,PH_END,PW_END,PL_END ∈ [-255, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1}
BatchNormalization | N/A, collapsed in graph optimization phase
BitwiseAnd | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseNot | input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseOr | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
BitwiseXor | lhs: Type: int8, int16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Cast | input: Type: int8, int16, bool8, int32 Shape: [*] output: Type: int8, int16, bool8, int32 Shape: [*]
Ceil | if qtype is int8 input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
otherwise input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Celu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Clip | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Concat | input: Type: No limits Arg Number: input number ∈ [1, 1024] Dim: all dims < 131072 Size: size < 2G output: The input and output types need to be the same. Dim: all dims < 131072 Size: size < 2G
Constant | N/A, collapsed in graph optimization phase
ConstantOfShape | N/A, collapsed in graph optimization phase
Conv | input: --conv 1d-- Type: int8, int16 Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; weight: --conv 1d-- Type: int8, int16 Shape: [N,KL,C] Dim: C ∈ [1, 8192]; KL ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KL × C ∈ [1, 65535] --conv 2d-- Type: int8, int16 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 31]; N ∈ [1, 1048576] if fout is the last layer of conv else [1, 8192] Size: KH × KW × C ∈ [1, 65535] --conv 3d Type: int8 Shape: [N,KD,KH,KW,C] N ∈ [1, 65535]; KD,KH,KW ∈ [1, 9]; Dim: C ∈ [1, 4096]; Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: --conv 1d-- Type: int8, int16, int32, float32, float16 Shape: [*,L,C] Dim: * ∈ [1, 4096]; L,C ∈ [1, 65536] --conv 2d-- Type: int8, int16, int32, float32, float16 Shape: [*,H,W,C] Dim: * ∈ [1, 4096]; H,W,C ∈ [1, 65536] --conv 3d-- Type: int8, int16, int32, float32, float16 Shape: [*,D,H,W,C] Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 4096]; stride: --conv 1d-- Shape: [SL] Dim: SL ∈ [1, 256]; SL ∈ {1} if dilation > 1 --conv 2d-- Shape: [SH,SW] Dim: SH,SW ∈ [1, 256]; SH,SW ∈ {1} if dilation > 1 --conv 3d-- Shape: [SD,SH,SW] Dim: SD,SH,SW must be 1 or 2 and equal to each other. pad: --conv 1d-- Shape: [P_left,P_right] Dim: P_left,P_right ∈ (-L, 256] --conv 2d-- Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ (-H, 256], P_left,P_right ∈ (-W, 256] --conv 3d-- Shape: [P_front, P_top, P_left, P_back, P_bottom, P_right] Dim: P_front,P_back ∈ [0, KD/2], P_top,P_bottom ∈ [0, KH/2], P_left,P_right ∈ [0, KW/2] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: --conv 1d-- Shape: [DL] Dim: DL ∈ [1, 18] --conv 2d-- Shape: [DH,DW] Dim: DH,DW ∈ [1, 18] --conv 3d-- Shape: [DD,DH,DW] DD,DH,DW = 1 others: --conv 1d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KL × fin.c' ∈ [1, 65535] --conv 2d-- Stride only support odd number and 2 when conv is a int16 depthwise conv If groupNum > 1, for each group, fin.c' ∈ [1, 65535], KH × KW × fin.c' ∈ [1, 65535] Fin.c' = fin.c × min(lcm(fout.c × (lcm(fin.c, 4) / fin.c), 8) / fout.c, groupNum)
ConvTranspose | input: --convtranspose 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [*,W,C] 1d_Dim: * ∈ [1, 128]; W ∈ [1, 65536]; C ∈ [1, 2048] 2d_Shape: [*,H,W,C] 2d_Dim: * ∈ [1, 128]; H,W ∈ [1, 65536]; C ∈ [1, 2048] --convtranspose 3d-- Type: int8 3d_Shape: [*,D,H,W,C] 3d_Dim: * ∈ [1, 128]; D,H,W ∈ [1, 65536]; C ∈ [1, 2048] weight: --convtranspose 1d/2d-- Type: int8, int16; input and weight cannot both be int16 1d_Shape: [N,KW,C] 1d_Dim: N,C ∈ [1, 2048]; KW ∈ [1, 14] 1d_Size: KW × C ∈ [1, 65535] 2d_Shape: [N,KH,KW,C] 2d_Dim: N,C ∈ [1, 2048]; KH,KW ∈ [1, 14]; KH,KW cannot both be 1 2d_Size: KH × KW × C ∈ [1, 65535] --convtranspose 3d-- Type: int8 3d_Shape: [N,KD,KH,KW,C] 3d_Dim: N,C ∈ [1, 2048]; KD,KH,KW ∈ [1, 14]; KD,KH,KW cannot all be 1 3d_Size: KD × KH × KW × C ∈ [1, 131072] bias: Type: float32 output: Type: int8, int16, int32, float16, float32 stride: 1d_Shape: [SW] 1d_Dim: SW ∈ [1, 14]; 2d_Shape: [SH,SW] 2d_Dim: SH,SW ∈ [1, 14]; 3d_Shape: [SD,SH,SW] 3d_Dim: SD,SH,SW ∈ [1, 14]; pad: 1d_Shape: [P_left,P_bottom] 1d_Dim: P_left,P_bottom ∈ [0, 256] 2d_Shape: [P_top,P_left,P_bottom,P_right] 2d_Dim: P_top,P_left,P_bottom,P_right ∈ [0, 256] 3d_Shape: [P_front,P_top,P_left,P_back,P_bottom,P_right] 3d_Dim: P_front,P_top,P_left,P_back,P_bottom,P_right ∈ [0, 256] groupNum: Fin.c is divisible by group number, conv 3d only support 1 dilation: 1d_Shape: [DW] 1d_Dim: DW ∈ {1} 2d_Shape: [DH,DW] 2d_Dim: DH,DW ∈ {1} 3d_Shape: [DD,DH,DW] 3d_Dim: DD,DH,DW ∈ {1}
Cos | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Cosh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
CumSum | input: Type: int8, int16 Shape: [*, dim[axis], *] Dim: * ∈ [1, 65536]; dim[axis] ∈ [1, 8192] output: Type: int8, int16, int32. only support combinations (int8→int8,int16,int32), (int16→int16,int32) The Shape and Dim is same as input
DeformConv | input: Type: int8 Shape: [*,H,W,C] Dim: H,W ∈ [1, 1024]; H × W ≤ 720 × 1024; other dims ∈ [1, 65536] offset: Type: int16 Shape: [*,OH,OW,2 × offsetGroupNum × KH × KW] Size: 2 × offsetGroupNum × KH × KW ∈ [2, 256], OH × KH × OW × KW ≤ 720 × 1024 mask: Type: int8 Shape: [*,OH,OW,offsetGroupNum × KH × KW] Size: offsetGroupNum × KH × KW ∈ [1, 128] The value of mask is usually [0, 1] weight: Type: int8 Shape: [N,KH,KW,C] Dim: C ∈ [1, 8192]; KH,KW ∈ [1, 8]; N ∈ [1, 4096] Size: KH × KW × C ∈ [1, 65536] bias: Type: float32 output: Type: int8, int16, int32 Other constraints: Same as fin stride: Shape: [SH,SW] Dim: SH,SW ∈ [1] pad: Shape: [P_top,P_left,P_bottom,P_right] Dim: P_top,P_bottom ∈ [-H/2, 256], P_left,P_right ∈ [-W/2, 256] groupNum: Fin.c is divisible by group number offsetGroupNum: Fin.c is divisible by offset group number Size: offsetGroupNum ∈ [1, 2] dilation: Shape: [DH,DW] Dim: DH,DW ∈ [1] others: For each group, fin.c ∈ [1, 8192], KH × KW × fin.c ∈ [1, 65535], fin.c = C when group = 1
DepthToSpace | input: Type: No limits output: The input and output types need to be the same.
Dropout | N/A, collapsed in graph optimization phase
Elu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Equal | lhs: Type: int8, int16, float16, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
Erf | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Exp | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Expand | input: Type: No limits output: The input and output types need to be the same.
EyeLike | input: Type: int8, int16, bool8, int32 Shape: [*] output: Type: int8, int16, bool8, int32 Shape: [*]
Floor | if qtype is int8 input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
otherwise input: Type: int8, int16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
GRU | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Type: The input and output types need to be the same.
Gather | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. index: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] index value should not be larger than 1048576 output: The input and output types need to be the same.
GatherElements | input: Type: int8, int16, int32, float16, float32 Shape: [*] Input will transpose to [N, W, C]. W is inputShape[dim], N is the product of inputShape[:dim], C is the product of inputShape[dim+1:]. N, C ∈ [1, 1048576]. N × C should not be larger than 1048576 When C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If input type is int32, float32, W ∈ [1, 65536]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*] indices value should not be larger than 32767 Indices will transpose to [N, D, C]. D is indicesShape[dim], N is the product of indicesShape[:dim], C is the product of indicesShape[dim+1:]. N, C ∈ [1, 1048576]. IndicesShape[i] <= inputShape[i] for all dimensions i != dim. output: The input and output types need to be the same.
GatherND | input: Type: int8, int16, int32, float16, float32 Shape: [*] If gather1d, W = inputShape[batchDim], when C <= 4, W ∈ [1, 1048576]; when C > 4, W ∈ [1, 131072]. If gather2d, H = inputShape[batchDim], W = inputShape[batchDim+1], H ∈ [1, 65536], W ∈ [1, 1048576]. If input type is int32, float32, W ∈ [1, 65536]. B is product of inputShape[0: batchDim], B ∈ [1, 1048576]. C is product of inputShape[batchDim+D:], C ∈ [1, 1048576]. indices: Type: int8, int16, int32; Unsupported negative indices. Shape: [*, D] indices value should not be larger than 1048576. D ∈ [1, 2]. output: Shape: [*] The input and output types need to be the same. batchDim: The number of batch dimensions. The gather of indexing starts from dimension of input[batchDim:]
Gelu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Gemm | lhs: Type: int8, int16 Shape: [M,K] Dim: M,K ∈ [1, 8192] rhl: Type: int8, int16 Shape: [K,N] Dim: K ∈ [1, 8192], N ∈ [1, 1048576] output: Type: int8, int16, int32, float16, float32 Shape: [M,N] Other constraints: Same as lhs and rhs
GlobalAveragePool | input: Type: int8, int16, float16 Shape: [*] Dim:  reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32) Shape: reduce dim will be 1 or fused depend on keepDim
GlobalMaxPool | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
Greater | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
GreaterOrEqual | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
GridSample | input: Type: int8 Shape: [*,H,W,C] Dim: H ∈ [1, 32768], W ∈ [1, 32768], other dims ∈ [1, 65536]. NOTE: H and W cannot both be greater than 4096 at the same time. grid: Type: int16 Shape: [*,H,W,2] output: The input and output types need to be the same except Dim constraints mode: Only support bilinear and nearest padding_mode: Only support zeros and border
GroupNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
HardSigmoid | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
HardSwish | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Identity | N/A, collapsed in graph optimization phase
InstanceNormalization | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
LSTM | input: Type: int8, int16 Dim: all dims < 2097152 size < 2G output: Type: The input and output types need to be the same.
LayerNormalization | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
LeakyRelu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Less | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
LessOrEqual | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
Log | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
LogSoftmax | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
MatMul | lhs: Type: int8, int16 Shape: [*,M,C] Dim: * ∈ [1, 4096], M,C ∈ [1, 8192] rhl: Type: int8, int16 Shape: [*,C,N] Dim: * ∈ [1, 4096]; C ∈ [1, 8192], N ∈  [1, 1048576] output: Type: int8, int16, int32, float16, float32 Shape: [*,M,N] Other constraints: Same as lhs and rhs
Max | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
MaxPool | input: Type: int8, int16, float16 Shape: [*,H,W,C] output: The input and output types need to be the same. kernel: Shape: [KL] or [KH,KW], only support 1d or 2d now Dim: 1d: KL ∈ [1, 256], KL*bitWidth/8 <= 24576; 2d: KH, KW ∈ [1, 256], KH*KW*bitWidth/8 <= 24576 stride: Shape: [SH,SW] Dim: SH, SW ∈ [1, 256] pad: Shape: [PH_BEGIN,PW_BEGIN,PH_END,PW_END] PH_BEGIN,PW_BEGIN,PH_END,PW_END ∈ [-4, 256] dilation: Shape: 1d: [DW]; 2d: [DH, DW] Dim: 1d: DW ∈ {1}; 2d: DH, DW ∈ {1}
Min | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Mish | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Mul | lhs: Type: int8, int16, float16 Shape: [*] rhs: Type: int8, int16, float16 Shape: [*] output: Type: int8, int16, int32, float16. only support (int8,int16→int8,int16,int32), (float16→float16). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Neg | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Not | input: Type: int8, int16, bool8 Shape: [*] output: Type: bool8 Shape: [*]
Or | lhs: Type: int8, int16, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]
PRelu | lhs: Type: int8, int16 Shape: [*] rhs: The type and shape is same as lhs output: The type and shape is same as lhs
Pad | N/A, collapsed in graph optimization phase
Pow | if exponent is an initializer with a single element 2, and the qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if exponent is an initializer with a single element 2, and the qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if exponent is an initializer with a single element 2, and the qtype is float16: input: Type: float16 output: Type: float16
Reciprocal | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
ReduceL2 | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
ReduceMax | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
ReduceMean | input: Type: int8, int16, float16 Shape: [*] Dim:  reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32) Shape: reduce dim will be 1 or fused depend on keepDim
ReduceMin | input: Type: int8, int16, float16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same. Shape: reduce dim will be 1 or fused depend on keepDim
ReduceSum | input: Type: int8, int16, float16 Shape: [*] Dim:  reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: int8, int16, float16, float32. only support combinations (int8→int8,int16), (int16→int16), (float16→float16,float32) Shape: reduce dim will be 1 or fused depend on keepDim
Relu | input: Type: int8, int16, int32, float16, float32, if type is int32 or float32, this op must be fusible to a Conv op Shape: [*] output: The input and output types need to be the same. Shape: [*]
Reshape | input: Type: No limits output: The input and output types need to be the same.
Resize | input: Type: int8, int16 Shape: [*,H,W,C] The integer part of step ∈ [-256, 255] output: Same as input mode: Support nearest and bilinear padValue: When padvalue is not equal to 0 and quantized, input only supports per tensor quantization
Round | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
ScatterElements | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: data and indices must be the same shape, except for scatterAxis updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min]
ScatterND | input: Type: float16, float32, if type is float32, scatterReduceMode must be add Shape: [*] indices: Type: int16, int32 Shape: [*,C] Dim: *, C ∈ [1, 2] updates: Type: float16 Shape: [*] output: The input and output types need to be the same. scatterReduceMode: Mode ∈ [none, add, max, min]
Selu | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Shape | N/A, collapsed in graph optimization phase
Sigmoid | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sign | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
Sin | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sinh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Size | N/A, collapsed in graph optimization phase
Slice | if starts/ends/axes/steps are constant: input: Type: No limits Dim: all dims < 2097152 output: Type: The output datatype must be the same as the input Dim: all dims < 2097152
Softmax | input: Type: int8, int16 Shape: [*] Dim: reduce axis dim size ∈ [1, 65535] Element : reduce Elements size ∈ [1, 65535] output: Type: The input and output types need to be the same.
Softplus | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
SpaceToDepth | input: Type: No limits output: The input and output types need to be the same.
Split | input: Type: No limits Dim: all dims < 2097152 output: The input and output types need to be the same. The input and output constraints are the same.
Sqrt | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Sub | lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Sum | lhs: Type: int8, int16, int32, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*] others: If type is int32, this op must be fusible to a conv op, one input is conv output which is int32, the other input must be int8 or int16. When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Tan | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
Tanh | if qtype is int8: input: Type: int8, int16 table: The table and input types need to be the same. output: The input and output types need to be the same.
if qtype is int16: input: Type: int16 range: Type: int16 Shape: [N + 2, 4] N: * ∈ [1, 64] entry: Type: int16 Shape: [N, 64] N: * ∈ [1, 64] output: Type: int16
if qtype is float16: input: Type: float16 output: Type: float16
ThresholdedRelu | input: Type: int8, int16, float16 Shape: [*] output: The input and output types need to be the same. Shape: [*]
Tile | input: Type: No limits output: The input and output types need to be the same.
TopK | input: Type: int8, int16, int32, float16, float32 output: The type of output value is same as input, and indices is integer type. k: K <= 1024 others: The combined size in bytes of the op's operands and outputs must be no more than 3.2MB.
Transpose | input: Type: No limits output: The input and output types need to be the same.
Trilu | lhs: Type: int8, int16, float16 Shape: [*] rhs: Type: int8, int16, float16 Shape: [*] output: Type: int8, int16, int32, float16. only support (int8,int16→int8,int16,int32), (float16→float16). Shape: [*] others: When inputs are of integer or quantized type, if the result overflows, it will saturate to the maximum or minimum value of the result data type.
Where | condition: Type: bool8 Shape: [*] lhs: Type: int8, int16, float16 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: The input and output types need to be the same. Shape: [*]
Xor | lhs: Type: int8, int16, bool8 Shape: [*] rhs: The lhs and rhs types need to be the same. Shape: [*] output: Type: bool8 Shape: [*]