微调 PaddleOCR-VL 后不能直接进行推理

[megemini]

Checklist:

• 查找历史相关issue寻求解答
• 翻阅FAQ
• 翻阅PaddleX 文档
• 确认bug是否在新版本里还未修复

描述问题

使用如下命令安装 paddlecor 相关组件进行推理：

python -m pip install -U "paddleocr[doc-parser]"
python -m pip install https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl
python -m pip install --force-reinstall opencv-python-headless
python -m pip install numpy==1.26.4

使用如下命令进行图片推理：

!python -m paddleocr doc_parser -i /home/aistudio/work/data/HUST-OBS/deciphered/0001/G_0001_後2.36.5合33075𠂤歷間.png \
    --vl_rec_model_name "PaddleOCR-VL-1.5-0.9B" \
    --vl_rec_model_dir "PaddleOCR-VL-15-SFT" \
    --save_path="PaddleOCR-VL-15-SFT_response" \
    --use_layout_detection=False \
    --prompt_label="ocr"

提示错误：

Checking connectivity to the model hosters, this may take a while. To bypass this check, set `PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK` to `True`.
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/utils/cpp_extension/extension_utils.py:712: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Creating model: ('PaddleOCR-VL-1.5-0.9B', 'PaddleOCR-VL-15-SFT')
Traceback (most recent call last):
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/__main__.py", line 39, in <module>
    console_entry()
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/__main__.py", line 26, in console_entry
    main()
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_cli.py", line 194, in main
    _execute(args)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_cli.py", line 183, in _execute
    args.executor(args)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 498, in execute_with_args
    perform_simple_inference(
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_utils/cli.py", line 62, in perform_simple_inference
    wrapper = wrapper_cls(**init_params)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 86, in __init__
    super().__init__(**kwargs)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_pipelines/base.py", line 67, in __init__
    self.paddlex_pipeline = self._create_paddlex_pipeline()
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddleocr/_pipelines/base.py", line 105, in _create_paddlex_pipeline
    return create_pipeline(config=self._merged_paddlex_config, **kwargs)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/pipelines/__init__.py", line 168, in create_pipeline
    pipeline = BasePipeline.get(pipeline_name)(
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/utils/deps.py", line 208, in _wrapper
    return old_init_func(self, *args, **kwargs)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py", line 113, in __init__
    self._pipeline = self._create_internal_pipeline(config, self.device)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/pipelines/_parallel.py", line 168, in _create_internal_pipeline
    return self._pipeline_cls(
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 140, in __init__
    self.vl_rec_model = self.create_model(vl_rec_config)
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/pipelines/base.py", line 106, in create_model
    model = create_predictor(
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/models/__init__.py", line 83, in create_predictor
    return BasePredictor.get(model_name)(
  File "/home/aistudio/external-libraries/lib/python3.10/site-packages/paddlex/inference/models/doc_vlm/predictor.py", line 59, in __init__
    raise RuntimeError("Static graph models are not supported")
RuntimeError: Static graph models are not supported

需要手动将微调后的模型 赋值并修改名称 /home/aistudio/PaddleOCR-VL-15-SFT/model-00001-of-00001.safetensors 修改为 /home/aistudio/PaddleOCR-VL-15-SFT/model.safetensors 才能正常推理。

也就是说，如果需要使用微调后的模型，需要再手动复制一遍模型文件。

复现

1. 您是否已经正常运行我们提供的教程？

2. 您是否在教程的基础上修改代码内容？还请您提供运行的代码

3. 您使用的数据集是？

4. 请提供您出现的报错信息及相关log

环境

1. 请提供您使用的PaddlePaddle和PaddleX的版本号

2. 请提供您使用的操作系统信息，如Linux/Windows/MacOS

3. 请问您使用的Python版本是？

4. 请问您使用的CUDA/cuDNN的版本号是？

[Bobholamovic]

PaddleX确实对模型名称有要求，不能是任意的。考虑到一个目录中很可能不只存在一个.safetensors文件，且规定文件名更加规范、易于处理，我建议是不是可以给ERNIEKit仓库提一个issue，将“重命名模型”的步骤加入到相关微调文档中？

[megemini]

PaddleX确实对模型名称有要求，不能是任意的。考虑到一个目录中很可能不只存在一个.safetensors文件，且规定文件名更加规范、易于处理，我建议是不是可以给ERNIEKit仓库提一个issue，将“重命名模型”的步骤加入到相关微调文档中？

同意！我来提！

[megemini]

@Bobholamovic 另外，PaddleX 这边是不是也有点问题？如果只是将 model-00001-of-00001.safetensors 改名为 model.safetensors，会报错：

File ~/external-libraries/lib/python3.10/site-packages/paddlex/inference/models/common/transformers/transformers/model_utils.py:259, in load_state_dict(checkpoint_file, tensor_parallel_split_mapping, fliter_dict_keys, device, ckpt_quant_stage, convert_from_hf, transpose_weight_keys)
    256 require_deps("safetensors")
    257 from safetensors import safe_open
--> 259 with safe_open(checkpoint_file, framework="paddle") as f:
    260     keys = list(f.keys())
    261 state_dict, scale_dict = _load_part_state_dict_from_safetensors(
    262     keys,
    263     checkpoint_file,
   (...)
    270     transpose_weight_keys=transpose_weight_keys,
    271 )

FileNotFoundError: No such file or directory: /home/aistudio/PaddleOCR-VL-15-SFT/model-00001-of-00001.safetensors

[Bobholamovic]

请问方便贴一下模型目录里有哪些文件吗？

[megemini]

请问方便贴一下模型目录里有哪些文件吗？

aistudio@jupyter-942478-10030707:~$ tree PaddleOCR-VL-15-SFT
PaddleOCR-VL-15-SFT
├── added_tokens.json
├── all_results.json
├── chat_template.jinja
├── checkpoint-180
│   ├── added_tokens.json
│   ├── config.json
│   ├── generation_config.json
│   ├── master_weights-00001-of-00001.safetensors
│   ├── master_weights.safetensors.index.json
│   ├── model-00001-of-00001.safetensors
│   ├── model.safetensors.index.json
│   ├── optimizer-00001-of-00001.safetensors
│   ├── optimizer.safetensors.index.json
│   ├── preprocessor_config.json
│   ├── rng_state_0.pth
│   ├── scheduler.pdparams
│   ├── special_tokens_map.json
│   ├── static_name_to_dyg_name.json
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   ├── trainer_state.json
│   └── training_args.bin
├── checkpoint-240
│   ├── added_tokens.json
│   ├── config.json
│   ├── generation_config.json
│   ├── master_weights-00001-of-00001.safetensors
│   ├── master_weights.safetensors.index.json
│   ├── model-00001-of-00001.safetensors
│   ├── model.safetensors.index.json
│   ├── optimizer-00001-of-00001.safetensors
│   ├── optimizer.safetensors.index.json
│   ├── preprocessor_config.json
│   ├── rng_state_0.pth
│   ├── scheduler.pdparams
│   ├── special_tokens_map.json
│   ├── static_name_to_dyg_name.json
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   ├── trainer_state.json
│   └── training_args.bin
├── checkpoint-300
│   ├── added_tokens.json
│   ├── config.json
│   ├── generation_config.json
│   ├── master_weights-00001-of-00001.safetensors
│   ├── master_weights.safetensors.index.json
│   ├── model-00001-of-00001.safetensors
│   ├── model.safetensors.index.json
│   ├── optimizer-00001-of-00001.safetensors
│   ├── optimizer.safetensors.index.json
│   ├── preprocessor_config.json
│   ├── rng_state_0.pth
│   ├── scheduler.pdparams
│   ├── special_tokens_map.json
│   ├── static_name_to_dyg_name.json
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   ├── trainer_state.json
│   └── training_args.bin
├── config.json
├── generation_config.json
├── inference.yml
├── model-00001-of-00001.safetensors
├── model.safetensors # 复制 model-00001-of-00001.safetensors 并改名为 model.safetensors
├── model.safetensors.index.json
├── preprocessor_config.json
├── special_tokens_map.json
├── static_name_to_dyg_name.json
├── tensorboard_logs
│   ├── events.out.tfevents.1771999632.jupyter-942478-10030707
│   └── vdlrecords.1771999631.log
├── tokenizer.model
├── tokenizer_config.json
├── train_results.json
├── trainer_state.json
└── training_args.bin

4 directories, 73 files

[Bobholamovic]

不确定是不是model-00001-of-00001.safetensors这个文件会干扰判断逻辑，建议可以试试把它移到别的目录去

[megemini]

不确定是不是model-00001-of-00001.safetensors这个文件会干扰判断逻辑，建议可以试试把它移到别的目录去

这个目录里面如果没有 model-00001-of-00001.safetensors 就会报上面的错误 FileNotFoundError: No such file or directory: /home/aistudio/PaddleOCR-VL-15-SFT/model-00001-of-00001.safetensors

[Bobholamovic]

会不会是某个配置文件里写死了这个路径？我建议可以在这个目录里检索一下model-00001-of-00001.safetensors，看看是否存在这样的内容

[megemini]

model.safetensors.index.json 里面有，也是 erniekit 那边一起修改了？