Rspress

# #Common Algorithm Model Samples

## #Sample Location

The common algorithmic model samples are located at `03_classification/`, `04_detection/` and `07_segmentation/` folders in the `horizon_model_convert_sample` path.

## #Prepare Datasets

### #Dataset Download Address

The dataset can be downloaded from the following address.

Dataset | Address
--- | ---
ImageNet | https://www.image-net.org/download.php
COCO | https://cocodataset.org/
VOC | http://host.robots.ox.ac.uk/pascal/VOC/ (need to download both versions 2007 and 2012)
Cityscapes | https://github.com/mcordts/cityscapesScripts

### #Dataset Reference Structure

To facilitate your subsequent steps, after the dataset has been downloaded, you need to process the evaluation dataset according to the structure suggested by Horizon below.

Dataset | Description
ImageNet Dataset | We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. `imagenet/ ├── calibration_data │ ├── ILSVRC2012_val_00000001.JPEG │ ├── ... │ └── ILSVRC2012_val_00000100.JPEG ├── ILSVRC2017_val.txt ├── val │ ├── ILSVRC2012_val_00000001.JPEG │ ├── ... │ └── ILSVRC2012_val_00050000.JPEG └── val.txt`
COCO Dataset | We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. `coco/ ├── calibration_data │ ├── COCO_val2014_000000181007.jpg │ ├── ... │ └── COCO_val2014_000000181739.jpg └── coco_val2017 ├── annotations │ ├── instances_train2017.json │ └── instances_val2017.json └── images ├── 000000000139.jpg ├── 000000000285.jpg ├── ... ├── 000000581615.jpg └── 000000581781.jpg`
VOC Dataset | Attention Please note that the VOC2012 directory currently stores two datasets, VOC2007 and VOC2012, so please follow the directory structure below for the evaluation dataset. Please contact Horizon if you encounter any problems during data preparation. `VOCdevkit/ └── VOC2012 ├── Annotations │ ├── 2007_000027.xml │ ├── ... │ └── 2012_004331.xml ├── ImageSets │ ├── Action │ │ ├── jumping_train.txt │ │ ├── jumping_trainval.txt │ │ ├── jumping_val.txt │ │ ├── ... │ │ ├── val.txt │ │ ├── walking_train.txt │ │ ├── walking_trainval.txt │ │ └── walking_val.txt │ ├── Layout │ │ ├── train.txt │ │ ├── trainval.txt │ │ └── val.txt │ ├── Main │ │ ├── aeroplane_train.txt │ │ ├── aeroplane_trainval.txt │ │ ├── aeroplane_val.txt │ │ ├── ... │ │ ├── train.txt │ │ ├── train_val.txt │ │ ├── trainval.txt │ │ ├── tvmonitor_train.txt │ │ ├── tvmonitor_trainval.txt │ │ ├── tvmonitor_val.txt │ │ └── val.txt │ └── Segmentation │ ├── train.txt │ ├── trainval.txt │ └── val.txt ├── JPEGImages │ ├── 2007_000027.jpg │ ├── ... │ └── 2012_004331.jpg ├── SegmentationClass │ ├── 2007_000032.png │ ├── ... │ └── 2011_003271.png ├── SegmentationObject │ ├── 2007_000032.png │ ├── ... │ └── 2011_003271.png └── train.txt`
Cityscapes Dataset | We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. `cityscapes/ ├── cityscapes_calibration_data │ ├── aachen_000000_000019_leftImg8bit.png │ ├── ... │ └── aachen_000099_000019_leftImg8bit.png ├── gtFine │ ├── test │ │ ├── berlin │ │ ├── ... │ │ └── munich │ ├── train │ │ ├── aachen │ │ ├── ... │ │ └── zurich │ └── val │ ├── frankfurt │ ├── lindau │ └── munster ├── leftImg8bit │ ├── test │ │ ├── berlin │ │ ├── ... │ │ └── munich │ ├── train │ │ ├── aachen │ │ ├── ... │ │ └── zurich │ └── val │ ├── frankfurt │ ├── lindau │ └── munster ├── license.txt └── README`

## #Prepare Models

When using the model conversion sample package, please prepare the corresponding floating-point model first.

Note

If you need to do this process in the sample folder, you need to execute the `00_init.sh` script in the folder first to get the corresponding original model and dataset.

## #Demo Of Algorithm Model Samples

Taking the `Resnet50` model as an example, this section illustrates the steps of floating-point to fixed-point model conversion by using the scripts in `03_classification/03_resnet50/`.

### #Docker Container Prep

First complete the docker installation and configuration and enter the docker container accoridng to Docker Container Deployment .

### #Obtain The Original Model And Calibration Dataset

Execute `00_init.sh` in the `03_classification/03_resnet50/` folder to obtain the model and calibration dataset needed for the current sample.

`# 1. Enter the folder the demo script resides cd samples/ai_toolchain/horizon_model_convert_sample/03_classification/03_resnet50/ # 2. Execute the script to obtain the original model and calibration dataset sh 00_init.sh`

### #Check If the Model is Executable

Run the script as follows:

`# Execute model checking sh 01_check.sh`

### #Prepare Calibration Dataset

Run the `02_preprocess.sh` script in the same directory, as follows:

`# Convert the images in 01_common/calibration_data/imagenet # to: ./calibration_data_rgb sh 02_preprocess.sh`
Note

- We extracted 100 images from the ImageNet dataset and use them as a calibration dataset. Before calibration, we pre-processed the data and convert short size resize/crop size/NHWC to NCHW/to rgb.
- The `hb_compile` tool will load data from the converted binary data, the format of the pre-processed binary data file is npy, the dtype is `unit8`.

### #Building Models

Run the `03_build.sh` script in the same directory, as follows:

`sh 03_build.sh`
Note

The above script uses the `hb_compile` tool to convert the model. The most important thing to focus on is the conversion configuration file, refer to Model Quantized Compilation section.

The output of the above script is as follows:

`ls model_output | cat resnet50_224x224_nv12.hbm resnet50_224x224_nv12_calibrated_model.onnx resnet50_224x224_nv12_ptq_model.onnx resnet50_224x224_nv12_optimized_float_model.onnx resnet50_224x224_nv12_original_float_model.onnx resnet50_224x224_nv12_quantized_model.bc resnet50_224x224_nv12.html resnet50_224x224_nv12.json resnet50_224x224_nv12_quant_info.json resnet50_224x224_nv12_advice.json resnet50_224x224_nv12_node_info.csv hb_compile.log`
Note

For now you ONLY need to focus on the resnet50_224x224_nv12.hbm file.

### #Single Image Inference

Run the `04_inference.sh` script to inference a single image, as follows:

`sh 04_inference.sh`
Note

- As image pre-processing, model data post-processing are required in the image inference, we offered a Python sample script, please refer to `sh 04_inference.sh`.
- This script is to perform the inference against a single image and verify whether the inference result meets the expectations. If you want to perform accuracy evaluation, refer to the script `05_evaluate.sh`.

### #Accuracy Evaluation

Run the `05_evaluate.sh` script to evaluate the accuracy, as follows:

`export PARALLEL_PROCESS_NUM=${parallel_process_num} sh 05_evaluate.sh`
Note

- As image pre-processing, model data post-processing are required in accuracy evaluation, awe offered a Python sample script, please refer to `sh 05_evaluate.sh`.
- To accelerate the evaluation, you can adjust the number of concurrent processes by using the `-p` option; meanwhile, pay attention to the memory usage. When the value of the `-p` option is set to 0 or left unfilled, the fixed-point models in the CPU environment will be processed by 10 processes, while other scenarios using 1 process.

## #FAQ

### #Why is the reproduced accuracy slightly different from the indicators in the documentation?

There are two possible reasons:

1. There may be minor differences in calculation methods when in different server environments, which can cause small data fluctuations in the accuracy of the compiled fixed-point ONNX models in different server environments when compared with the documentation.
2. The different versions of third-party libraries such as OpenCV and NumPy, which may produce different results after pre-processing, and this may also lead to slight data fluctuations in reproduced accuracy when compared with the documentation.

No need to worry much about this situation, the records provided in the documentation is only for reference, and it is ok that your reproduced accuracy is slightly different from those in documents.

### #Why is the fixed-point model accuracy not aligned with the on-board accuracy of the hbm file in the ai_benchmark sample?

In the standard delivery, when adding the sample, we have already aligned the fixed-point model accuracy with the hbm on-board accuracy in the ai_benchmark sample.

If you find any unaligned accuracy, we recommend you first checking the model input for consistency.

When executing the fixed-point model evaluation script, you use the dataset of image type, while for the hbm model used on board, you use the binary dataset converted by the hb_eval_preprocess tool.

Based on this, if the dataset you used on-board is not generated by using the above methods, we recommend that you first use our data preprocessing tool (i.e., hb_eval_preprocess) to regenerate the dataset needed for on-board running on the same server that you run the fixed-point model accuracy and rerun the on-board accuracy to ensure the model inputs are consistent.

Attention

Make sure to use the same environment to generate the dataset by using the hb_eval_preprocess tool and to run the fixed-point model accuracy.

On This Page