refactor: update backend options and enhance documentation for hybrid parsing methods

This commit is contained in:
myhloli
2025-12-25 19:32:06 +08:00
parent 984b303dfa
commit bf61e022d8
2 changed files with 39 additions and 11 deletions

View File

@@ -10,11 +10,11 @@ Options:
-v, --version Show version and exit
-p, --path PATH Input file path or directory (required)
-o, --output PATH Output directory (required)
-m, --method [auto|txt|ocr] Parsing method: auto (default), txt, ocr (pipeline backend only)
-b, --backend [pipeline|vlm-transformers|vlm-vllm-engine|vlm-lmdeploy-engine|vlm-http-client]
Parsing backend (default: pipeline)
-m, --method [auto|txt|ocr] Parsing method: auto (default), txt, ocr (pipeline and hybrid* backend only)
-b, --backend [pipeline|hybrid-auto-engine|hybrid-http-client|vlm-auto-engine|vlm-http-client]
Parsing backend (default: hybrid-auto-engine)
-l, --lang [ch|ch_server|ch_lite|en|korean|japan|chinese_cht|ta|te|ka|th|el|latin|arabic|east_slavic|cyrillic|devanagari]
Specify document language (improves OCR accuracy, pipeline backend only)
Specify document language (improves OCR accuracy, pipeline and hybrid* backend only)
-u, --url TEXT Service address when using http-client
-s, --start INTEGER Starting page number for parsing (0-based)
-e, --end INTEGER Ending page number for parsing (0-based)
@@ -45,8 +45,8 @@ Options:
files to be input need to be placed in the
`example` folder within the directory where
the command is currently executed.
--enable-vllm-engine BOOLEAN Enable vllm engine backend for faster
processing.
--enable-http-client BOOLEAN Enable http-client backend to link openai-
compatible servers.
--enable-api BOOLEAN Enable gradio API for serving the
application.
--max-convert-pages INTEGER Set the maximum number of pages to convert
@@ -111,3 +111,17 @@ Here are the environment variables and their descriptions:
- `MINERU_INTER_OP_NUM_THREADS`:
* Used to set the inter_op thread count for ONNX models, affects the parallel execution of multiple operators
* Default is `-1` (auto-select), can be set to other values via environment variable to adjust the thread count.
- `MINERU_HYBRID_BATCH_RATIO`:
* Used to set the batch ratio for small model processing in `hybrid-*` backends.
* Commonly used in `hybrid-http-client`, it allows adjusting the VRAM usage of a single client by controlling the batch ratio of small models.
* Single Client VRAM Size | MINERU_HYBRID_BATCH_RATIO
------------------------|--------------------------
<= 6 GB | 8
<= 4.5 GB | 4
<= 3 GB | 2
<= 2.5 GB | 1
- `MINERU_HYBRID_FORCE_PIPELINE_ENABLE`:
* Used to force the text extraction part in `hybrid-*` backends to be processed using small models.
* Defaults to `false`. Can be set to `true` via environment variable to enable this feature, thereby reducing hallucinations in certain extreme cases.

View File

@@ -10,11 +10,11 @@ Options:
-v, --version 显示版本并退出
-p, --path PATH 输入文件路径或目录(必填)
-o, --output PATH 输出目录(必填)
-m, --method [auto|txt|ocr] 解析方法auto默认、txt、ocr仅用于 pipeline 后端)
-b, --backend [pipeline|vlm-transformers|vlm-vllm-engine|vlm-lmdeploy-engine|vlm-http-client]
解析后端(默认为 pipeline
-m, --method [auto|txt|ocr] 解析方法auto默认、txt、ocr仅用于 pipeline 与 hybrid* 后端)
-b, --backend [pipeline|hybrid-auto-engine|hybrid-http-client|vlm-auto-engine|vlm-http-client]
解析后端(默认为 hybrid-auto-engine
-l, --lang [ch|ch_server|ch_lite|en|korean|japan|chinese_cht|ta|te|ka|th|el|latin|arabic|east_slavic|cyrillic|devanagari]
指定文档语言(可提升 OCR 准确率,仅用于 pipeline 后端)
指定文档语言(可提升 OCR 准确率,仅用于 pipeline 与 hybrid* 后端)
-u, --url TEXT 当使用 http-client 时,需指定服务地址
-s, --start INTEGER 开始解析的页码(从 0 开始)
-e, --end INTEGER 结束解析的页码(从 0 开始)
@@ -43,7 +43,7 @@ Usage: mineru-gradio [OPTIONS]
Options:
--enable-example BOOLEAN 启用示例文件输入(需要将示例文件放置在当前
执行命令目录下的 `example` 文件夹中)
--enable-vllm-engine BOOLEAN 启用 vllm 引擎后端以提高处理速度
--enable-http-client BOOLEAN 在后端选项中启用 HTTP 客户端选项
--enable-api BOOLEAN 启用 Gradio API 以提供应用程序服务
--max-convert-pages INTEGER 设置从 PDF 转换为 Markdown 的最大页数
--server-name TEXT 设置 Gradio 应用程序的服务器主机名
@@ -106,3 +106,17 @@ MinerU命令行工具的某些参数存在相同功能的环境变量配置
- `MINERU_INTER_OP_NUM_THREADS`
* 用于设置onnx模型的inter_op线程数影响多个算子的并行执行
* 默认为`-1`(自动选择),可通过环境变量设置为其他值以调整线程数。
- `MINERU_HYBRID_BATCH_RATIO`
* 用于设置 hybrid-* 后端中 小模型处理的batch倍率
* 在hybrid-http-client中较为常用可以通过控制小模型的batch倍率来调整单个客户端的显存占用量
* 单个client端显存大小 | MINERU_HYBRID_BATCH_RATIO
------------------|------------------------
<= 6 GB | 8
<= 4.5 GB | 4
<= 3 GB | 2
<= 2.5 GB | 1
- `MINERU_HYBRID_FORCE_PIPELINE_ENABLE`
* 用于强制将 hybrid-* 后端中的 文本提取部分使用 小模型 进行处理
* 默认为`false`,可通过环境变量设置为`true`来启用该功能,从而在某些极端情况下减少幻觉的发生。