1# RKNNToolkit2 OPs Support 2 3## Explanation of terms: 4 5**Remarks**: 6 7 Operators' specifications must meet the remarks' requirements. 8 9**Broadcast rule**: 10 11- per-layer： 12 13 shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar ==> shape(result) = (2, 3, 4, 5) 14 15- per-channel: 16 17 shape(A) = (2, 3, 4, 5), shape(B) = (3,), ==> shape(result) = (2, 3, 4, 5) 18 19 shape(A) = (2, 3, 4, 5), shape(B) = (1,3,1,1), ==> shape(result) = (2, 3, 4, 5) 20 21- per-element: 22 23 shape(A) = (2, 3, 4, 5), shape(B) = (2,3,4,5) ==> shape(result) = (2, 3, 4, 5) 24 25- other: 26 27 shape(A) = (2, 3, 4, 5), shape(B) = (5,) ==> shape(result) = (2, 3, 4, 5) 28 29**Input Size Restrictions Description** 30 31 32Assuming that input size is [N,H,W,C] (layout is NHWC) 33 34- Case 1: the first layer is **Convolution**, whose kernel size is [kernel_height, kernel_width] 35 36 **W * kernel_height < 7168** 37 38 **kernel_height * kernel_width < 128** 39 40 41- Case 2: first layer is not Convolution, and C == 1 or C == 3 or C == 4 42 43 **W < 7168** 44 45- others: 46 47 **No Restrictions** 48 49 50 51 52 53 54## ONNX OPs supported by RKNN Toolkit2 55 56According to [ONNX official instructions](https://github.com/microsoft/onnxruntime/blob/master/docs/Versioning.md 'ONNX Version Description'), the corresponding ONNX opset version is 12. 57The list of ONNX OPs supported by RKNN Toolkit2 is as follows: 58<br>(For more restrictions, please refer to [RKNN_Compiler_Support_Operator_List.pdf](https://github.com/rockchip-linux/rknpu2/tree/master/doc)) 59 60| **Operators** | **Remarks** | 61| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 62| Abs | Not Supported | 63| Acos | Not Supported | 64| Acosh | Not Supported | 65| Add | | 66| And | Not Supported | 67| ArgMax | | 68| ArgMin | | 69| Asin | Not Supported | 70| Asinh | Not Supported | 71| Atan | Not Supported | 72| Atanh | Not Supported | 73| AveragePool | **NPU Limit:**<br />channel: [1, 8192]<br />stride height/width: [1, 8]<br />pad left/right/top/bottom: [0, 7]<br />auto_pad: NOTSET<br />count_include_pad: 1 <br />ceil_mode: 0 | 74| BatchNormalization | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 75| BitShift | Not Supported | 76| Cast | only support bool/int8/float | 77| Ceil | Not Supported | 78| Celu | Not Supported | 79| Clip | | 80| Compress | Not Supported | 81| Concat | | 82| ConcatFromSequence | Not Supported | 83| Constant | | 84| ConstantOfShape | | 85| Conv | **NPU Limit:**<br />kernel height/width: [1, 31]<br />stride height/width: [1, 7]<br />pad left/right/top/bottom: [0, 15] | 86| ConvInteger | Not Supported | 87| ConvTranspose | **NPU Limit:**<br />kernel height/width: [1, 31]<br />stride height/width: 2, 4, 8<br />pad left/right/top/bottom: [0, 15] | 88| Cos | Not Supported | 89| Cosh | Not Supported | 90| CumSum | Not Supported | 91| DepthToSpace | | 92| DequantizeLinear | | 93| Det | | 94| Div | **NPU Limit:**<br />support broadcast rule: per-element/other | 95| Dropout | | 96| Einsum | Not Supported | 97| Elu | channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176]<br /> | 98| Equal | | 99| Erf | Not Supported | 100| Exp | | 101| Expand | Not Supported | 102| EyeLike | only support constant input | 103| Flatten | | 104| Floor | Not Supported | 105| GRU | batchsize: 1 | 106| Gather | | 107| GatherElements | Not Supported | 108| GatherND | Not Supported | 109| Gemm | | 110| GlobalAveragePool | channel: [1, 8192]<br />kernel height/width: [1, 343]<br /> | 111| GlobalLpPool | Not Supported | 112| GlobalMaxPool | channel: [1, 8192]<br />kernel height/width: [1, 343]<br /> | 113| Greater | **NPU Limit:**<br />support broadcast rule: per-element/other | 114| GreaterOrEqual | | 115| HardSigmoid | | 116| HardSwish | | 117| Hardmax | Not Supported | 118| Identity | | 119| If | only support constant input | 120| InstanceNormalization | | 121| IsInf | Not Supported | 122| IsNaN | Not Supported | 123| LRN | | 124| LSTM | batchsize: 1<br />input_forget: 0 | 125| LeakyRelu | | 126| Less | **NPU Limit:**<br />support broadcast rule: per-element/other | 127| LessOrEqual | | 128| Log | Not Supported | 129| LogSoftmax | batchsize: 1 | 130| Loop | Not Supported | 131| LpNormalization | | 132| LpPool | Not Supported | 133| MatMul | | 134| MatMulInteger | Not Supported | 135| Max | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 136| MaxPool | **NPU Limit:**<br />channel: [1, 8192]<br />stride height/width: [1, 8]<br />pad left/right/top/bottom: [0, 7]<br />auto_pad: NOTSET<br />ceil_mode: 0<br />dilations: 1<br />storage_order: 0 | 137| MaxRoiPool | | 138| MaxUnpool | | 139| Mean | Not Supported | 140| Min | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 141| Mod | Not Supported | 142| Mul | **NPU Limit:**<br />support broadcast rule: per-layer/channel/element | 143| Multinomial | Not Supported | 144| Neg | Not Supported | 145| NonMaxSuppression | Not Supported | 146| NonZero | Not Supported | 147| Not | Not Supported | 148| OneHot | Not Supported | 149| Or | Not Supported | 150| PRelu | slope support broadcast rule: per-layer/channel | 151| Pad | **NPU Limit:**<br />width: [1, 8176]<br />mode: constant<br />pads n_begin/n_end/c_begin/c_end: 1 | 152| Pow | | 153| QLinearConv | Not Supported | 154| QLinearMatMul | Not Supported | 155| QuantizeLinear | | 156| RNN | Not Supported | 157| RandomNormal | Not Supported | 158| RandomNormalLike | Not Supported | 159| RandomUniform | Not Supported | 160| RandomUniformLike | Not Supported | 161| Range | Not Supported | 162| Reciprocal | Not Supported | 163| ReduceL1 | Not Supported | 164| ReduceL2 | Not Supported | 165| ReduceLogSum | Not Supported | 166| ReduceLogSumExp | Not Supported | 167| ReduceMax | | 168| ReduceMean | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8192] | 169| ReduceMin | | 170| ReduceProd | Not Supported | 171| ReduceSum | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8192] | 172| ReduceSumSquare | | 173| Relu | | 174| Reshape | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 175| Resize | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176]<br />scales: [1, 8] | 176| ReverseSequence | | 177| RoiAlign | pool type: average<br />batchsize: 1 | 178| Round | Not Supported | 179| Scan | Not Supported | 180| ScatterElements | Not Supported | 181| ScatterND | Not Supported | 182| Selu | Not Supported | 183| SequenceAt | Not Supported | 184| SequenceConstruct | Not Supported | 185| SequenceEmpty | Not Supported | 186| SequenceErase | Not Supported | 187| SequenceInsert | Not Supported | 188| SequenceLength | Not Supported | 189| Shape | | 190| Shrink | Not Supported | 191| Sigmoid | | 192| Sign | Not Supported | 193| Sin | Not Supported | 194| Sinh | Not Supported | 195| Size | | 196| Slice | batchsize: 1<br />**NPU Limit:**<br />steps: 1 | 197| Softmax | batchsize: 1<br />**NPU Limit:**<br />channel: [1, 8192]<br />axis: 1 | 198| Softplus | | 199| Softsign | Not Supported | 200| SpaceToDepth | | 201| Split | | 202| SplitToSequence | Not Supported | 203| Sqrt | | 204| Squeeze | | 205| StringNormalizer | Not Supported | 206| Sub | **NPU Limit:**<br />support broadcast rule: per-layer/channel/element | 207| Sum | Not Supported | 208| Tan | Not Supported | 209| Tanh | | 210| TfIdfVectorizer | Not Supported | 211| ThresholdedRelu | Not Supported | 212| Tile | batchsize: 1<br />not support broadcast | 213| TopK | Not Supported | 214| Transpose | **NPU Limit:**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 215| Trilu | Not Supported | 216| Unique | Not Supported | 217| Unsqueeze | | 218| Where | | 219| Xor | Not Supported | | | 220 221 222## Caffe OPs supported by RKNN Toolkit2 223 224Caffe protocols RKNN Toolkit2 uses only based on the officially modified protocol of berkeley. 225The protocol based on the official revision of berkeley comes from [berkeley caffe](https://github.com/BVLC/caffe/tree/master/src/caffe/proto 'Berkeley Caffe'), commit hash is 21d0608. On this basis RKNN Toolkit2 have added some OPs. 226Based on this protocol, the list of Caffe OPs supported by RKNN Toolkit2 is as follows: 227 228| **Operators** | **Remarks** | 229| ---------------------- | ------------------------------------------------------------------------------------------------------------- | 230| BatchNorm | same as onnx BatchNormalization | 231| bn (BatchNorm + Scale) | same as onnx BatchNormalization according to https://github.com/TimoSaemann/caffe-segnet-cudnn5 | 232| BNLL | | 233| Concat | same as onnx Concat | 234| Convolution | same as onnx Conv | 235| ConvolutionDepthwise | kernel height/width: [1, 8]<br />others same as onnx Conv | 236| Crop | | 237| Deconvolution | same as ConvTranspose | 238| Dropout | | 239| Eltwise | support broadcast rule: per-layer/channel/element | 240| Flatten | | 241| HardSigmoid | | 242| InnerProduct | same as onnx Gemm | 243| LRN | same as onnx LRN | 244| Lstm | same as onnx LSTM according to https://github.com/xmfbit/warpctc-caffe | 245| Normalize | | 246| Permute | same as onnx Transpose | 247| Power | | 248| Pooling | same as onnx pooling | 249| PRelu | same as onnx PRelu | 250| Proposal | batch: 1 | 251| Reduction | output dims <= 4 | 252| Relu | same as onnx Relu | 253| Relu6 | same as onnx Clip | 254| Reorg | | 255| Reshape | same as onnx Reshape | 256| Resize | bilinear; nearest | 257| Reverse | | 258| ROIPooling | same as MaxRoiPool according to https://github.com/twmht/caffe-pva-faster-rcnn | 259| Scale | same as onnx Mul | 260| Sigmoid | same as onnx Sigmoid | 261| Slice | same as onnx Split | 262| Softmax | same as onnx Softmax | 263| Split | same as onnx Slice | 264| TanH | same as onnx TanH | 265| Tile | same as onnx Tile | 266| Transpose | same as onnx Transpose | 267| Upsample | according to https://github.com/SeanQ88/caffe_upsample and https://github.com/TimoSaemann/caffe-segnet-cudnn5 | 268 269 270## Pytorch OPs supported by RKNN Toolkit2 271 272The Pytorch version supported by RKNN Toolkit2 is >1.6.0, models generated by other versions may not support. 273The list of Pytorch OPs supported by RKNN Toolkit2 is as follows: 274 275| **Operators** | **Remarks** | 276| ----------------------------- | ---------------------------------------------------------------------------------- | 277| aten::_convolution | same as onnx Conv | 278| aten::abs | Not supported | 279| aten::abs_ | Not supported | 280| aten::adaptive_avg_pool1d | Not supported | 281| aten::adaptive_avg_pool2d | same as onnx AveragePool | 282| aten::adaptive_max_pool1d | Not supported | 283| aten::adaptive_max_pool2d | same as onnx MaxPool | 284| aten::add | same as onnx Add | 285| aten::add_ | | 286| aten::addmm | same as onnx Gemm | 287| aten::affine_grid_generator | Not supported | 288| aten::alpha_dropout | | 289| aten::alpha_dropout_ | Not supported | 290| aten::arange | Not supported | 291| aten::avg_pool1d | Not supported | 292| aten::avg_pool2d | same as onnx AveragePool | 293| aten::avg_pool3d | Not supported | 294| aten::batch_norm | same as onnx BatchNormalization | 295| aten::bmm | same as onnx MatMul | 296| aten::cat | same as onnx Concat | 297| aten::celu | Not supported | 298| aten::celu_ | Not supported | 299| aten::chunk | | 300| aten::clamp | | 301| aten::clamp_ | | 302| aten::clamp_max | Not supported | 303| aten::clamp_max_ | Not supported | 304| aten::clamp_min | Not supported | 305| aten::clamp_min_ | Not supported | 306| aten::clone | | 307| aten::constant_pad_nd | same as onnx Pad | 308| aten::contiguous | | 309| aten::copy | | 310| aten::cos | Not supported | 311| aten::cos_ | Not supported | 312| aten::cumsum | Not supported | 313| aten::detach | | 314| aten::detach_ | Not supported | 315| aten::div | same as onnx Div | 316| aten::div_ | | 317| aten::dropout | | 318| aten::dropout_ | | 319| aten::einsum | Not supported | 320| aten::elu | same as onnx Elu | 321| aten::elu_ | | 322| aten::embedding | same as onnx Gather | 323| aten::empty | | 324| aten::eq | Not supported | 325| aten::eq_ | Not supported | 326| aten::erf | Not supported | 327| aten::erf_ | Not supported | 328| aten::erfc | Not supported | 329| aten::erfc_ | Not supported | 330| aten::exp | | 331| aten::exp_ | | 332| aten::expand | Not supported | 333| aten::expand_as | Not supported | 334| aten::expm1 | Not supported | 335| aten::expm1_ | Not supported | 336| aten::feature_dropout | | 337| aten::feature_dropout_ | Not supported | 338| aten::flatten | | 339| aten::floor | Not supported | 340| aten::floor_ | Not supported | 341| aten::floor_divide | Not supported | 342| aten::floor_divide_ | Not supported | 343| aten::gather | Not supported | 344| aten::ge | Not supported | 345| aten::ge_ | Not supported | 346| aten::gelu | | 347| aten::gelu_ | Not supported | 348| aten::grid_sampler | Not supported | 349| aten::gru | | 350| aten::gt | | 351| aten::gt_ | Not supported | 352| aten::hardshrink | Not supported | 353| aten::hardshrink_ | Not supported | 354| aten::hardswish | same as onnx HardSwish | 355| aten::hardswish_ | | 356| aten::hardtanh | | 357| aten::hardtanh_ | | 358| aten::index | Not supported | 359| aten::index_put | Not supported | 360| aten::index_put_ | Not supported | 361| aten::instance_norm | same as onnx InstanceNormalization | 362| aten::Int | | 363| aten::layer_norm | **NPU Limit**<br />channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8192] | 364| aten::le | Not supported | 365| aten::le_ | Not supported | 366| aten::leaky_relu | same as onnx LeakyRelu | 367| aten::leaky_relu_ | | 368| aten::lerp | Not supported | 369| aten::lerp_ | Not supported | 370| aten::log | Not supported | 371| aten::log_ | Not supported | 372| aten::log10 | Not supported | 373| aten::log10_ | Not supported | 374| aten::log1p | Not supported | 375| aten::log1p_ | Not supported | 376| aten::log2 | Not supported | 377| aten::log2_ | Not supported | 378| aten::log_sigmoid | Not supported | 379| aten::log_softmax | Not supported | 380| aten::linear | same as onnx Gemm | 381| aten::lstm | same as onnx LSTM | 382| aten::lt | | 383| aten::lt_ | Not supported | 384| aten::matmul | same as onnx MatMul | 385| aten::max | | 386| aten::max_ | Not supported | 387| aten::max_pool1d | same as onnx MaxPool | 388| aten::max_pool1d_with_indices | | 389| aten::max_pool2d | same as onnx MaxPool | 390| aten::max_pool2d_with_indices | | 391| aten::mean | same as onnx ReduceMean | 392| aten::meshgrid | Not supported | 393| aten::min | | 394| aten::min_ | Not supported | 395| aten::mm | same as onnx MatMul | 396| aten::mul | same as onnx Mul | 397| aten::mul_ | | 398| aten::narrow | same as onnx Slice | 399| aten::ne | Not supported | 400| aten::ne_ | Not supported | 401| aten::neg | Not supported | 402| aten::neg_ | Not supported | 403| aten::new_full | Not supported | 404| aten::new_zeros | Not supported | 405| aten::nonzero | Not supported | 406| aten::norm | Not supported | 407| aten::ones | | 408| aten::ones_like | | 409| aten::pad | Not supported | 410| aten::permute | same as onnx Transpose | 411| aten::pow | | 412| aten::pow_ | Not supported | 413| aten::prelu | same as onnx PRelu | 414| aten::prelu_ | Not supported | 415| aten::reciprocal | | 416| aten::reciprocal_ | Not supported | 417| aten::reflection_pad1d | | 418| aten::reflection_pad2d | | 419| aten::relu | same as onnx Relu | 420| aten::relu_ | | 421| aten::repeat | | 422| aten::reshape | | 423| aten::reshape_ | Not supported | 424| torchvision::roi_align | Not supported | 425| aten::rsqrt | Not supported | 426| aten::rsqrt_ | Not supported | 427| aten::ScalarImplicit | | 428| aten::select | | 429| aten::selu | Not supported | 430| aten::selu_ | Not supported | 431| aten::sigmoid | same as onnx Sigmoid | 432| aten::sigmoid_ | | 433| aten::silu | | 434| aten::silu_ | | 435| aten::sin | Not supported | 436| aten::sin_ | Not supported | 437| aten::size | | 438| aten::slice | same as onnx Slice | 439| aten::softmax | same as onnx Softmax | 440| aten::softplus | | 441| aten::softshrink | Not supported | 442| aten::sort | Not supported | 443| aten::split | same as onnx Split | 444| aten::split_with_sizes | | 445| aten::sqrt | Not supported | 446| aten::sqrt_ | Not supported | 447| aten::squeeze | | 448| aten::squeeze_ | Not supported | 449| aten::stack | | 450| aten::sub | same as onnx Sub | 451| aten::sub_ | | 452| aten::sum | same as onnx ReduceSum | 453| aten::t | | 454| aten::t_ | Not supported | 455| aten::tanh | | 456| aten::tanh_ | | 457| aten::threshold | | 458| aten::threshold_ | | 459| aten::to | | 460| aten::topk | Not supported | 461| aten::transpose | | 462| aten::transpose_ | | 463| aten::true_divide | same as onnx Div | 464| aten::true_divide_ | Not supported | 465| aten::type_as | | 466| aten::unfold | Not supported | 467| aten::unsqueeze | | 468| aten::upsample_bilinear2d | | 469| aten::upsample_nearest2d | | 470| aten::view | | 471| aten::view_ | Not supported | 472| aten::view_as | Not supported | 473| aten::view_as_ | Not supported | 474| aten::zero_ | Not supported | 475| aten::zeros | | 476| aten::zeros_like | | 477 478## TensorFlow OPs supported by RKNN Toolkit2 479 480The pb files (contain OPs belows) generated by TensorFlow version 1.12 - 1.15 for 1.x and 2.3 - 2.5 for 2.x are supported by RKNN Toolkit2. For more information on TensorFlow version compatibility, please refer to [tensorflow official instructions on OP version](https://www.tensorflow.org/guide/versions 'Tensorflow official instructions on OP version') . 481The list of TensorFlow OPs supported by RKNN Toolkit2 is as follows: 482 483| **Operators** | **Remarks** | 484| --------------------- | --------------------------------------------------------------- | 485| Add | same as onnx Add | 486| AvgPool | same as onnx AveragePool | 487| Concat | same as onnx Concat | 488| Conv2D | same as onnx Conv | 489| DepthToSpace | | 490| DepthwiseConv2d | kernel height/width: [1, 8]<br />others same as onnx Conv | 491| Div | same as onnx Div | 492| Dropout | | 493| Flatten | | 494| LeakyRelu | same as onnx LeakyRelu | 495| Less | same as onnx Less | 496| LRN | | 497| MatMul | | 498| MaxPool | same as onnx MaxPool | 499| Mean | output dims <= 4 | 500| Pad | same as onnx Pad | 501| Relu | same as onnx Relu | 502| Reshape | | 503| ResizeBilinear | | 504| ResizeNearestNeighbor | | 505| Sigmoid | | 506| Slice | | 507| Softmax | | 508| Softplus | channel: [1, 8192]<br />height: [1, 8192]<br />width: [1, 8176] | 509| SpaceToDepth | | 510| Split | | 511| Squeeze | | 512| StridedSlice | | 513| Tanh | same as onnx TanH | 514| Transpose | | 515 516## Darknet OPs supported by RKNN Toolkit2 517The list of Darknet OPs supported by RKNN Toolkit2 is as follows: 518 519| **Operators** | **Remarks** | 520| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 521| add | same as onnx Add | 522| batchnormalize | same as onnx BatchNormalization | 523| concat | same as onnx Concat | 524| convolutional | same as onnx Conv | 525| depthwise_convolutional | kernel height/width: [1, 8]<br />others same as onnx Conv | 526| fullconnect | | 527| leakyrelu | same as onnx LeakyRelu | 528| mish | | 529| pooling | **AveragePool**: same as onnx AveragePool <br /> **GlobalAveragePool**: same as onnx GlobalAveragePool <br /> **MaxPool/GlobalMaxPool**: same as onnx MaxPool/GlobalMaxPool | 530| route | | 531| shortcut | | 532| softmax | | 533| upsampling | |