Compass Unified Parser is designed to convert models of various frameworks into a floating-point intermediate representation (IR). This IR is a standard IR designed by ARM China for the neural network compilation of the Zhouyi series. device.

Parser’s processing flow and design philosophy

The main goal of Parser is to convert a trained model into a floating-point IR and feed it to OPT (optimizer). The following is the main processing flow of Parser:

1. A model is passed to Parser through a unified configuration file.
2. The configuration parser parses the configuration file and submits a task to the corresponding model reader according to different configurations.
3. A supported model reader will take over the input model and complete the reading of the model:
   • Parse the model file (eg protobuf/flattenbuf/json or some proprietary format) and build a raw graph representation.
   • Convert the nodes in the original graph representation to internal unified nodes, for example:
     • Merge several TensorFlow nodes into one GRUSeq node.
     • will caffe detectionoutput Node converted to detectbox and nms node.
4. The model reader generates an internal unified graph representation, which is then passed to the front-end optimizer.
5. The main operation object of the front-end optimizer is the unified graph representation. It will merge or eliminate some nodes, for example:
   • mergeconv and add to a node.
   • merge conv/fc node and batchnorm node to node.
   • Eliminate some useless nodes, for example: a swap dimension unchangedtransposenode.
6. After optimization, Parser will perform at least one shape inference to obtain the shapes of all tensors.
7. Do some extra processing, such as:
   • Add some postprocessing nodes for some models.
8. Serialized to an IR file.

Graph and Node the design of

Inside Parser, similar to other frameworks, we use Graph and Node To represent a model, use a linked list to represent a graph. Graph Only all nodes are saved, and the topological relationship between nodes is saved in a Node and another Nodeconnect. Node To represent the layer concept (layer) in IR, Node can be accessed by calling serialize method to serialize into a string.

About Parser Design

• Parser only supports a graph with a fixed shape (static graph), and will perform several shape derivations during the entire parsing and conversion process.
• After every graph operation, such as merging, transforming, and eliminating nodes, we hope to perform a shape derivation, unless you know and guarantee that all shapes are correct.
  • Shape inference is done because graph operations may change the graph topology and may also result in changes in shape.
  • If some parameters depend on the shape, then please put the processing of these parameters after the shape derivation or in the derivation phase.
• The optimized processing only points out the unified graph representation, and does not support the original graph representation. Therefore, models from all frameworks can benefit from these optimizations.

quick start

installation guide

Parser is part of the Compass AIPUBuilder (NN-Compiler) compiler. You can refer to the following Compass AIPUBuilder guide to install AIPUBuilder. After completing the installation of AIPUBuilder, Parser will also be installed and can be used directly.

You can also passCompass_IntegrationInstructions in to compile an AIPUBuilder that includes a Parser.For instructions on using AIPUBuilder, please refer toMiniPkgThe manual inside: Zhouyi_Compass_Software_Programming_Guide_61010011_0205_01_en.pdf.

In addition to this, Parser can run alone.As long as the following dependencies are met, it can be run directly main.pyfile to run Parser.

install dependencies

• python (3.8 or higher)
• numpy
• onnx (> 12)
• protobuf
• flatbuffers
• tensorflow (== 2.6)
• torch

Run Parser

Parser is driven by the configuration file as input, you can use the following example to run Parser

python3 main.py -c my_config.ini

Configuration file format

All options must be in Common Inside the section:

• input_shape [required]

  The shape of the input tensor. Regular models have only one input tensor, such as:input_shape=[1,224,224,3] If you have multiple input tensors, separate them with commas, such as: input_shape=[1,224,224,3],[1,112,112,3]

• model_name [required]

  Enter a name for the model

• model_type [optional]

  The framework of the input model, the default is tensorflow, currently supports:

  • tensorflow
  • tflite
  • onnx
  • caffe

• model_domain [required]

  Classification of models, for example:

  • image_classification
  • object_detection
  • keyword_spotting
  • speech_recognition

• detection_postprocess [required 当 model_domain 是 object_detection]

  If your model isobject_detectionand you are using the official model, you can choose the following two post-processing methods, we will add the corresponding post-processing nodes at the end:

  • caffe_fasterrcnn
  • ssd
  • ssd_resnet
  • yolo2
  • yolo3_tiny
  • yolo3_full

• input_model [required]

  The file path of the input model, currently supports tensorflow frozen pb, tflite, caffe and onnx formats.

• input [required]

  The name of the input node (or tensor), if there are multiple inputs, use English commas,separated.

• output [required]

  Output node (or tensor) name, if there are multiple outputs, use English commas,separated.

Configuration file example

[Common]
input_shape = [1,224,224,3]
model_name = resnet50
model_domain = image_classification
detection_postprocess =
input_model = resnet50/frozen.pb
input = Placeholder
output = resnet_v1_50/predictions/Reshape

For more examples please refer to examples.

Run the example

First, you need to download the corresponding original model.you can passexamplesThe download_model.sh script below to download.

sh examples/tensorflow/download_model.sh

Then configure the corresponding input and output in the example.cfg file

[Common]
model_type = tensorflow
model_name = gru_l
model_domain = image_classification
input_model = ./GRU_L.pb
input = Mfcc:0
input_shape = [1, 49, 10]
output = labels_softmax:0
output_dir = ./

run run_example.py

python3 run_example.py --framework [specify example] --input_data [specify feed data]

Contribution Guidelines