保存模型时提示“RuntimeError: record_file is empty, no layers to be quantized”
问题描述
训练后量化场景,量化因子记录文件中无法找到某层的shift_bit字段,无法进行后续量化模型保存动作,量化流程终止。提示信息如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
Traceback (most recent call last): File "./GenAIModel/amt/tools/tmp_amct_tools.py", line 457, in wrapper func(*args, **kwargs) File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 1806, in test_AMCT37_TF_D16W8_AVGPOOL_003 run_case(config, acc=True) File "/raid/AMCT/demo/new_project/test/test_tf_iter37_d16w8.py", line 130, in run_case res = tf_api_compress(model, input_data, quant_cfg=quant_cfg,data_config=data_config, nuq=nuq, per_channel=per_channel) File "./GenAIModel/amt/model_process/infer_quant.py", line 754, in tf_api_compress save_model(model, out_puts, record_file, f'{save_path}/{case_name}' ) File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/common/utils/check_params.py", line 43, in wrapper return func(*args, **kwargs) File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 68, in save_model save_model_inner(pb_model, outputs, record_file, save_path) File "/raid/AMCT/daily/daily_env/lib/python3.9/site-packages/amct_tensorflow/interface/save_model.py", line 102, in save_model_inner raise RuntimeError( RuntimeError: record_file is empty, no layers to be quantized. please ensure calibration is finished by checking information! |
可能原因
网络的尾层节点为量化节点时,如果量化推理时不修改推理的输出节点,则会导致该问题。此场景下,调用quantize_model时,会有warning信息提示输出节点发生变化。
处理建议
该场景下调用quantize_model时,会有warning信息提示输出节点发生变化,在执行校准的过程中,根据提示信息,修改输出节点即可。
父主题: FAQ