mirror of
https://github.com/opendatalab/MinerU.git
synced 2026-03-28 03:28:34 +07:00
Compare commits
15 Commits
release-2.
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
61248e2ec9 | ||
|
|
c717a1c83a | ||
|
|
daf970af0e | ||
|
|
077b3101b3 | ||
|
|
a12610fb3e | ||
|
|
53aad4c900 | ||
|
|
345c46a457 | ||
|
|
e460f33c95 | ||
|
|
e9091876b6 | ||
|
|
c68dc3682a | ||
|
|
40796b9a7e | ||
|
|
31122e655b | ||
|
|
3eef5157f8 | ||
|
|
5cc95f3760 | ||
|
|
e31c0ec34d |
35
README.md
35
README.md
@@ -57,6 +57,7 @@
|
||||
- [Cambricon](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Cambricon/)
|
||||
- [Kunlunxin](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Kunlunxin/)
|
||||
- [Tecorigin](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Tecorigin/)
|
||||
- [Biren](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Biren/)
|
||||
- MinerU continues to support domestic hardware platforms and mainstream chip architectures. With secure and reliable technology, it helps research, government, and enterprise users reach new heights in document digitization!
|
||||
|
||||
- 2026/01/30 2.7.4 Release
|
||||
@@ -318,24 +319,25 @@ Currently, some models in this project are trained based on YOLO. However, since
|
||||
# Citation
|
||||
|
||||
```bibtex
|
||||
@misc{niu2025mineru25decoupledvisionlanguagemodel,
|
||||
title={MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing},
|
||||
author={Junbo Niu and Zheng Liu and Zhuangcheng Gu and Bin Wang and Linke Ouyang and Zhiyuan Zhao and Tao Chu and Tianyao He and Fan Wu and Qintong Zhang and Zhenjiang Jin and Guang Liang and Rui Zhang and Wenzheng Zhang and Yuan Qu and Zhifei Ren and Yuefeng Sun and Yuanhong Zheng and Dongsheng Ma and Zirui Tang and Boyu Niu and Ziyang Miao and Hejun Dong and Siyi Qian and Junyuan Zhang and Jingzhou Chen and Fangdong Wang and Xiaomeng Zhao and Liqun Wei and Wei Li and Shasha Wang and Ruiliang Xu and Yuanyuan Cao and Lu Chen and Qianqian Wu and Huaiyu Gu and Lindong Lu and Keming Wang and Dechen Lin and Guanlin Shen and Xuanhe Zhou and Linfeng Zhang and Yuhang Zang and Xiaoyi Dong and Jiaqi Wang and Bo Zhang and Lei Bai and Pei Chu and Weijia Li and Jiang Wu and Lijun Wu and Zhenxiang Li and Guangyu Wang and Zhongying Tu and Chao Xu and Kai Chen and Yu Qiao and Bowen Zhou and Dahua Lin and Wentao Zhang and Conghui He},
|
||||
year={2025},
|
||||
eprint={2509.22186},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CV},
|
||||
url={https://arxiv.org/abs/2509.22186},
|
||||
@article{dong2026minerudiffusion,
|
||||
title={MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding},
|
||||
author={Dong, Hejun and Niu, Junbo and Wang, Bin and Zeng, Weijun and Zhang, Wentao and He, Conghui},
|
||||
journal={arXiv preprint arXiv:2603.22458},
|
||||
year={2026}
|
||||
}
|
||||
|
||||
@misc{wang2024mineruopensourcesolutionprecise,
|
||||
title={MinerU: An Open-Source Solution for Precise Document Content Extraction},
|
||||
author={Bin Wang and Chao Xu and Xiaomeng Zhao and Linke Ouyang and Fan Wu and Zhiyuan Zhao and Rui Xu and Kaiwen Liu and Yuan Qu and Fukai Shang and Bo Zhang and Liqun Wei and Zhihao Sui and Wei Li and Botian Shi and Yu Qiao and Dahua Lin and Conghui He},
|
||||
year={2024},
|
||||
eprint={2409.18839},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CV},
|
||||
url={https://arxiv.org/abs/2409.18839},
|
||||
@article{niu2025mineru2,
|
||||
title={Mineru2. 5: A decoupled vision-language model for efficient high-resolution document parsing},
|
||||
author={Niu, Junbo and Liu, Zheng and Gu, Zhuangcheng and Wang, Bin and Ouyang, Linke and Zhao, Zhiyuan and Chu, Tao and He, Tianyao and Wu, Fan and Zhang, Qintong and others},
|
||||
journal={arXiv preprint arXiv:2509.22186},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{wang2024mineru,
|
||||
title={Mineru: An open-source solution for precise document content extraction},
|
||||
author={Wang, Bin and Xu, Chao and Zhao, Xiaomeng and Ouyang, Linke and Wu, Fan and Zhao, Zhiyuan and Xu, Rui and Liu, Kaiwen and Qu, Yuan and Shang, Fukai and others},
|
||||
journal={arXiv preprint arXiv:2409.18839},
|
||||
year={2024}
|
||||
}
|
||||
|
||||
@article{he2024opendatalab,
|
||||
@@ -358,6 +360,7 @@ Currently, some models in this project are trained based on YOLO. However, since
|
||||
|
||||
|
||||
# Links
|
||||
- [MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding](https://github.com/opendatalab/MinerU-Diffusion)
|
||||
- [Easy Data Preparation with latest LLMs-based Operators and Pipelines](https://github.com/OpenDCAI/DataFlow)
|
||||
- [Vis3 (OSS browser based on s3)](https://github.com/opendatalab/Vis3)
|
||||
- [LabelU (A Lightweight Multi-modal Data Annotation Tool)](https://github.com/opendatalab/labelU)
|
||||
|
||||
@@ -57,6 +57,7 @@
|
||||
- [寒武纪 Cambricon](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Cambricon/)
|
||||
- [昆仑芯 Kunlunxin](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Kunlunxin/)
|
||||
- [太初元碁 Tecorigin](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Tecorigin/)
|
||||
- [壁仞 Biren](https://opendatalab.github.io/MinerU/zh/usage/acceleration_cards/Biren/)
|
||||
- MinerU 持续兼容国产硬件平台,支持主流芯片架构。以安全可靠的技术,助力科研、政企用户迈向文档数字化新高度!
|
||||
|
||||
- 2026/01/30 2.7.4 发布
|
||||
@@ -325,24 +326,18 @@ mineru -p <input_path> -o <output_path> -b pipeline
|
||||
# Citation
|
||||
|
||||
```bibtex
|
||||
@misc{niu2025mineru25decoupledvisionlanguagemodel,
|
||||
title={MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing},
|
||||
author={Junbo Niu and Zheng Liu and Zhuangcheng Gu and Bin Wang and Linke Ouyang and Zhiyuan Zhao and Tao Chu and Tianyao He and Fan Wu and Qintong Zhang and Zhenjiang Jin and Guang Liang and Rui Zhang and Wenzheng Zhang and Yuan Qu and Zhifei Ren and Yuefeng Sun and Yuanhong Zheng and Dongsheng Ma and Zirui Tang and Boyu Niu and Ziyang Miao and Hejun Dong and Siyi Qian and Junyuan Zhang and Jingzhou Chen and Fangdong Wang and Xiaomeng Zhao and Liqun Wei and Wei Li and Shasha Wang and Ruiliang Xu and Yuanyuan Cao and Lu Chen and Qianqian Wu and Huaiyu Gu and Lindong Lu and Keming Wang and Dechen Lin and Guanlin Shen and Xuanhe Zhou and Linfeng Zhang and Yuhang Zang and Xiaoyi Dong and Jiaqi Wang and Bo Zhang and Lei Bai and Pei Chu and Weijia Li and Jiang Wu and Lijun Wu and Zhenxiang Li and Guangyu Wang and Zhongying Tu and Chao Xu and Kai Chen and Yu Qiao and Bowen Zhou and Dahua Lin and Wentao Zhang and Conghui He},
|
||||
year={2025},
|
||||
eprint={2509.22186},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CV},
|
||||
url={https://arxiv.org/abs/2509.22186},
|
||||
@article{niu2025mineru2,
|
||||
title={Mineru2. 5: A decoupled vision-language model for efficient high-resolution document parsing},
|
||||
author={Niu, Junbo and Liu, Zheng and Gu, Zhuangcheng and Wang, Bin and Ouyang, Linke and Zhao, Zhiyuan and Chu, Tao and He, Tianyao and Wu, Fan and Zhang, Qintong and others},
|
||||
journal={arXiv preprint arXiv:2509.22186},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@misc{wang2024mineruopensourcesolutionprecise,
|
||||
title={MinerU: An Open-Source Solution for Precise Document Content Extraction},
|
||||
author={Bin Wang and Chao Xu and Xiaomeng Zhao and Linke Ouyang and Fan Wu and Zhiyuan Zhao and Rui Xu and Kaiwen Liu and Yuan Qu and Fukai Shang and Bo Zhang and Liqun Wei and Zhihao Sui and Wei Li and Botian Shi and Yu Qiao and Dahua Lin and Conghui He},
|
||||
year={2024},
|
||||
eprint={2409.18839},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CV},
|
||||
url={https://arxiv.org/abs/2409.18839},
|
||||
@article{wang2024mineru,
|
||||
title={Mineru: An open-source solution for precise document content extraction},
|
||||
author={Wang, Bin and Xu, Chao and Zhao, Xiaomeng and Ouyang, Linke and Wu, Fan and Zhao, Zhiyuan and Xu, Rui and Liu, Kaiwen and Qu, Yuan and Shang, Fukai and others},
|
||||
journal={arXiv preprint arXiv:2409.18839},
|
||||
year={2024}
|
||||
}
|
||||
|
||||
@article{he2024opendatalab,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 基础镜像配置 vLLM 或 LMDeploy ,请根据实际需要选择其中一个,要求 amd64(x86-64) CPU + Cambricon MLU.
|
||||
# Base image containing the LMDEPLOY inference environment, requiring amd64(x86-64) CPU + Cambricon MLU.
|
||||
FROM crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/camb:qwen2.5_vl
|
||||
FROM crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/camb:mineru25
|
||||
ARG BACKEND=lmdeploy
|
||||
# Base image containing the vLLM inference environment, requiring amd64(x86-64) CPU + Cambricon MLU.
|
||||
# FROM crpi-vofi3w62lkohhxsp.cn-shanghai.personal.cr.aliyuncs.com/opendatalab-mineru/mlu:vllm0.8.3-torch2.6.0-torchmlu1.26.1-ubuntu22.04-py310
|
||||
@@ -39,4 +39,4 @@ RUN /bin/bash -c '\
|
||||
WORKDIR /workspace
|
||||
|
||||
# Set the entry point to activate the virtual environment and run the command line tool
|
||||
ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]
|
||||
ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]
|
||||
|
||||
112
docs/zh/usage/acceleration_cards/Biren.md
Normal file
112
docs/zh/usage/acceleration_cards/Biren.md
Normal file
@@ -0,0 +1,112 @@
|
||||
## 1. 测试平台
|
||||
以下为本指南测试使用的平台信息,供参考:
|
||||
```
|
||||
os: Ubuntu 22.04.4 LTS
|
||||
cpu: Intel x86-64
|
||||
gpu: Biren 106C
|
||||
driver: 1.10.0
|
||||
docker: 28.0.4
|
||||
```
|
||||
|
||||
## 2. 环境准备
|
||||
|
||||
### 2.1 下载并加载镜像 (vllm)
|
||||
|
||||
```bash
|
||||
wget http://birentech.com/xxx/MinerU/mineru-vllm.tar 链接获取请联系壁仞内部人员(邮箱:MonaLiu@birentech.com)
|
||||
docker load -i mineru-vllm.tar
|
||||
```
|
||||
|
||||
## 3. 启动 Docker 容器
|
||||
|
||||
```bash
|
||||
docker run -it --name mineru_docker \
|
||||
--privileged \
|
||||
--network=host \
|
||||
--shm-size=100G \
|
||||
-e MINERU_MODEL_SOURCE=local \
|
||||
-e MINERU_DEVICE_MODEL=supa \
|
||||
-e SHAPE_TRANSFORM_GRANK=true \
|
||||
mineru:biren-vllm-latest \
|
||||
/bin/bash
|
||||
```
|
||||
|
||||
|
||||
执行该命令后,您将进入到Docker容器的交互式终端,您可以直接在容器内运行MinerU相关命令来使用MinerU的功能。
|
||||
您也可以直接通过替换`/bin/bash`为服务启动命令来启动MinerU服务,详细说明请参考[通过命令启动服务](https://opendatalab.github.io/MinerU/zh/usage/quick_usage/#apiwebuihttp-clientserver)。
|
||||
|
||||
|
||||
## 4. 注意事项
|
||||
|
||||
不同环境下,MinerU对Biren加速卡的支持情况如下表所示:
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2" colspan="2">使用场景</th>
|
||||
<th colspan="2">容器环境</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>vllm</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="3">命令行工具(mineru)</td>
|
||||
<td>pipeline</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-auto-engine</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-http-client</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3">fastapi服务(mineru-api)</td>
|
||||
<td>pipeline</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-auto-engine</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-http-client</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3">gradio界面(mineru-gradio)</td>
|
||||
<td>pipeline</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-auto-engine</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><vlm/hybrid>-http-client</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="2">openai-server服务(mineru-openai-server)</td>
|
||||
<td>🟢</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="2">数据并行 (--data-parallel-size)</td>
|
||||
<td>🔴</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
注:
|
||||
🟢: 支持,运行较稳定,精度与Nvidia GPU基本一致
|
||||
🟡: 支持但较不稳定,在某些场景下可能出现异常,或精度存在一定差异
|
||||
🔴: 不支持,无法运行,或精度存在较大差异
|
||||
|
||||
>[!TIP]
|
||||
> - Biren加速卡指定可用加速卡的方式与NVIDIA GPU类似,请参考[使用指定GPU设备](https://opendatalab.github.io/MinerU/zh/usage/advanced_cli_parameters/#cuda_visible_devices)章节说明,
|
||||
>将环境变量`CUDA_VISIBLE_DEVICES`替换为`SUPA_VISIBLE_DEVICES`即可。
|
||||
> - 在壁仞平台可以通过`brsmi`命令查看加速卡的使用情况,并根据需要指定空闲的加速卡ID以避免资源冲突。
|
||||
@@ -19,8 +19,9 @@
|
||||
* [寒武纪 Cambricon](acceleration_cards/Cambricon.md) 🚀
|
||||
* [昆仑芯 Kunlunxin](acceleration_cards/Kunlunxin.md) 🚀
|
||||
* [太初元碁 Tecorigin](acceleration_cards/Tecorigin.md) ❤️
|
||||
* [AMD](acceleration_cards/AMD.md) [#3662](https://github.com/opendatalab/MinerU/discussions/3662) ❤️
|
||||
* [瀚博 VastAI](acceleration_cards/VastAI.md) [#4237](https://github.com/opendatalab/MinerU/discussions/4237)❤️
|
||||
* [壁仞 Biren](acceleration_cards/Biren.md) ❤️
|
||||
* [AMD #3662](https://github.com/opendatalab/MinerU/discussions/3662) ❤️
|
||||
* [瀚博 VastAI #4237](https://github.com/opendatalab/MinerU/discussions/4237) ❤️
|
||||
- 插件与生态
|
||||
* [Cherry Studio](plugin/Cherry_Studio.md)
|
||||
* [Sider](plugin/Sider.md)
|
||||
|
||||
@@ -1 +1 @@
|
||||
__version__ = "2.7.5"
|
||||
__version__ = "2.7.6"
|
||||
|
||||
Reference in New Issue
Block a user