下载
中文
注册

模型中存在不支持量化的层,量化模型失败

问题现象

执行ATC模型转换命令时,通过--compression_optimize_conf参数配置模型量化(将模型中的权重由浮点数float32量化到低比特整数int8)相关的选项,结果报错提示如下:

ATC start working now, please wait for a moment.
[ERROR][ProcessScale][52] Not support scale greater than 1 / FLT_EPSILON.
[ERROR][WtsArqCalibrationCpuKernel][188] ArqQuantCPU scale is illegal.
[ERROR][ArqQuant][301] WtsArqCalibrationCpuKernel of format CO_CI_KH_KW failed.
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[weight_algorithm.cpp:137]Default/network-DeepLabV3/resnet-Resnet/layer4-SequentialCell/0-Bottleneck/downsample-SequentialCell/0-Conv2d/Conv2D-op311 arq weight fake quant failed!
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[weight_calibration_pass.cpp:90]Fail to excute WeightFakeQuant without trans!
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[weight_calibration_pass.cpp:185]layer Default/network-DeepLabV3/resnet-Resnet/layer4-SequentialCell/0-Bottleneck/downsample-SequentialCell/0-Conv2d/Conv2D-op311 run WeightFakeQuantArq failed
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[graph_optimizer.cpp:43]pass run failed
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[quantize_api.cpp:227]Do GenerateCalibrationGraph optimizer pass failed.
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:19[quantize_api.cpp:363]Generate calibration Graph failed.
[ERROR] AMCT(14815,atc.bin):2023-04-14-12:23:22[inner_graph_calibration.cpp:78]Failed to excute InnerQuantizeGraph failed.

解决办法

通过报错提示layer xxxxxx run WeightFakeQuantArq failed可知,当前模型中有权重相关的层不支持量化,可以通过配置跳过不支持量化的层。

  1. 增加配置,跳过不支持量化的层。

    新增一个配置文件,文件名后缀为.cfg,例如simple_config.cfg,文件内容如下,加粗部分为报错提示中不支持量化的层:

    skip_layers:"Default/network-DeepLabV3/resnet-Resnet/layer4-SequentialCell/0-Bottleneck/downsample-SequentialCell/0-Conv2d/Conv2D-op311"

    同时,在--compression_optimize_conf参数指定的量化配置文件中,增加config_file参数:

    calibration:
    {
        input_data_dir: xxxxxx
        config_file: simple_config.cfg
        input_shape: xxxxxx
        infer_soc: xxxxxx
    }
  2. 重新执行模型转换。
  3. 重新执行推理。

    如果跳过不支持量化的层影响模型推理的结果数据,则需要用户自行调整模型,再重新量化模型。